PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Private Types | Private Member Functions | Private Attributes
PLearn::ProcessSymbolicSequenceVMatrix Class Reference

This VMatrix takes a VMat of a sequence of symbolic elements (corresponding to a set of symbolic attributes) and constructs context rows. More...

#include <ProcessSymbolicSequenceVMatrix.h>

Inheritance diagram for PLearn::ProcessSymbolicSequenceVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::ProcessSymbolicSequenceVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 ProcessSymbolicSequenceVMatrix (bool call_build_=false)
 Source VMat, from which contexts are extracted.
 ProcessSymbolicSequenceVMatrix (VMat s, int l_context, int r_context, bool call_build_=true)
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transform a shallow copy into a deep copy.
virtual real getStringVal (int col, const string &str) const
 returns value associated with a string (or MISSING_VALUE if there's no association for this string)
virtual string getValString (int col, real val) const
 returns the string associated with value val for field# col.
virtual void getValues (int row, int col, Vec &values) const
 Gives the possible values for a certain field in the VMatrix.
virtual void getValues (const Vec &input, int col, Vec &values) const
 Gives the possible values of a certain field (column) given the input.
virtual void getExample (int i, Vec &input, Vec &target, real &weight)
 Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.
virtual PP< DictionarygetDictionary (int col) const
 Return the Dictionary object for a certain field, or a null pointer if there isn't one.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual
ProcessSymbolicSequenceVMatrix
deepCopy (CopiesMap &copies) const

Static Public Member Functions

static string _classname_ ()
 Declare name and deepCopy methods.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

int n_left_context
 Number of elements at the left of the target element.
int n_right_context
 Number of elements at the right of the target element.
int conditions_offset
 Offset for the position of the element on which conditions are tested (default = 0).
bool conditions_for_exclusion
 Indication that the specified conditions are for the exclusion (true) or inclusion (false) of elements in the VMatrix.
bool full_context
 Indication that ignored elements of context should be replaced by the next nearest valid element.
bool put_only_target_attributes
 Indication that the only target fields of the VMatrix rows should be the (target) attributes of the context's target element.
bool use_last_context
 Indication that the last accessed context should be put in a buffer.
bool exclude_missing_target_tokens
 Indication to exclude from the VMatrix the context centered around a token with missing value target fields.
TVec< TVec< pair< int, int > > > conditions
 Conditions to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.
TVec< TVec< pair< int, string > > > string_conditions
 Conditions, in string format, to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.
TVec< TVec< pair< int, int > > > delimiters
 Delimiters of context.
TVec< TVec< pair< int, string > > > string_delimiters
 Delimiters, in string format, of context.
TVec< TVec< pair< int, int > > > ignored_context
 Elements to be ignored in context.
TVec< TVec< pair< int, string > > > string_ignored_context
 Elements, in string format, to be ignored in context.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 Fill the vector 'v' with the content of the i-th row.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Private Types

typedef SourceVMatrix inherited
typedef hash_map< int, intthis_hash_map

Private Member Functions

void build_ ()
 This does the actual building.
void from_string_to_int_format (TVec< TVec< pair< int, string > > > &str_format, TVec< TVec< pair< int, int > > > &int_format)
void from_string_to_int_format (TVec< pair< int, string > > &str_format, TVec< pair< int, int > > &int_format)
bool is_true (const TVec< TVec< pair< int, int > > > &conditions, const Vec row) const
void fill_current_row (int i, int &cp, int &target_position) const
 Fills the current_row_i field, and returns the number of non missing values for the ith context.

Private Attributes

bool fixed_context
 Indication that the context is of fixed length.
int max_context_length
 Maximum length of a context.
int n_attributes
 Number of attributes.
int current_target_pos
 Position in the last context of the target element (symbol and attributes)
int lower_bound
 Position in the source sequence of the first element in the last context.
int upper_bound
 Position in the source sequence of the last element in the last context.
Vec current_row_i
 Last context.
this_hash_map current_context_pos
 Mapping from the position in the source sequence to the position in the last context.
TVec< intindices
 Indices of the selected contexts.
TVec< intleft_positions
 Temporary fields.
TVec< intright_positions
Vec left_context
 Temporary fields.
Vec right_context
Vec row
Vec element
Vec target_element
Vec subinput

Detailed Description

This VMatrix takes a VMat of a sequence of symbolic elements (corresponding to a set of symbolic attributes) and constructs context rows.

An example of a sequence of elements would be a sequence of words, with their lemma form and POS tag. This sequence is encoded using integers, and is represented by the source VMatrix such as each row is an element, and each column is a certain type of attribute (e.g. lemma, POS tag, etc.). The source VMat string mapping (functions getStringVal(...) and getValString(...)) contains the integer/string encoding of the symbolic data. The context rows can be of fixed length, or constrained by delimiter symbols. Certain rows can be selected/excluded, and certain elements can be excluded of a context according to some conditions on its attributes. The conditions are expressed as disjunctions of conjunctions. For example:

[ [ 0 : "person", 1 : "NN" ] , [ 0 : "kind", 2 : "plural" ] ]

is equivalent in C++ logic form to:

(fields[0]=="person" && fields[1]=="NN") || (fields[0]=="kind" && fields[2]=="plural")

Conditions can be expressed in string or int format. The integer/string mapping is used to make the correspondance. We call the 'target element' of a context the element around which other elements are collected to construct the context.

Definition at line 84 of file ProcessSymbolicSequenceVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 89 of file ProcessSymbolicSequenceVMatrix.h.

Definition at line 91 of file ProcessSymbolicSequenceVMatrix.h.


Constructor & Destructor Documentation

PLearn::ProcessSymbolicSequenceVMatrix::ProcessSymbolicSequenceVMatrix ( bool  call_build_ = false)

Source VMat, from which contexts are extracted.

Definition at line 51 of file ProcessSymbolicSequenceVMatrix.cc.

References build_().

    : inherited(call_build_),
      n_left_context(2), n_right_context(2),
      conditions_offset(0), conditions_for_exclusion(1), full_context(true),
      put_only_target_attributes(false), use_last_context(true), 
      exclude_missing_target_tokens(false)
    /* ### Initialise all fields to their default value */
{
    if( call_build_ )
        build_();
}

Here is the call graph for this function:

PLearn::ProcessSymbolicSequenceVMatrix::ProcessSymbolicSequenceVMatrix ( VMat  s,
int  l_context,
int  r_context,
bool  call_build_ = true 
)

Definition at line 63 of file ProcessSymbolicSequenceVMatrix.cc.

References build_(), n_left_context, and n_right_context.

    : inherited(s, call_build_),
      conditions_offset(0), conditions_for_exclusion(1), full_context(true),
      use_last_context(true)
    /* ### Initialise all fields to their default value */
{
    n_left_context= l_context;
    n_right_context = r_context;
    if( call_build_ )
        build_();
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::ProcessSymbolicSequenceVMatrix::_classname_ ( ) [static]

Declare name and deepCopy methods.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

OptionList & PLearn::ProcessSymbolicSequenceVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

RemoteMethodMap & PLearn::ProcessSymbolicSequenceVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

bool PLearn::ProcessSymbolicSequenceVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

Object * PLearn::ProcessSymbolicSequenceVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

StaticInitializer ProcessSymbolicSequenceVMatrix::_static_initializer_ & PLearn::ProcessSymbolicSequenceVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

void PLearn::ProcessSymbolicSequenceVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::SourceVMatrix.

Definition at line 388 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::SourceVMatrix::build(), and build_().

Here is the call graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 247 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::TVec< T >::append(), PLearn::TVec< T >::clear(), conditions, conditions_for_exclusion, conditions_offset, current_context_pos, current_row_i, current_target_pos, delimiters, element, exclude_missing_target_tokens, fixed_context, from_string_to_int_format(), full_context, i, ignored_context, indices, PLearn::VMatrix::inputsize_, PLearn::is_missing(), is_true(), j, left_context, left_positions, PLearn::TVec< T >::length(), PLearn::VMatrix::length(), PLearn::VMat::length(), PLearn::VMatrix::length_, lower_bound, max_context_length, n_attributes, n_left_context, n_right_context, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), right_context, right_positions, row, PLearn::SourceVMatrix::source, string_conditions, string_delimiters, string_ignored_context, target_element, PLearn::VMatrix::targetsize_, PLearn::tostring(), PLearn::VMatrix::updateMtime(), upper_bound, PLearn::VMatrix::weightsize_, PLearn::VMat::width(), and PLearn::VMatrix::width_.

Referenced by build(), and ProcessSymbolicSequenceVMatrix().

{

    if(!source)
        PLERROR("In SelectAttributeSequenceVMatrix::build_() : no source"
                " defined");

    //  defineSizes(source->inputsize(),source->targetsize(),source->weightsize()); pas bon car ecrase declare options
    n_attributes = source->width();
    row.resize(n_attributes);
    element.resize(n_attributes);
    target_element.resize(n_attributes);
    max_context_length = 0;

    fixed_context = n_left_context >=0 && n_right_context >= 0;

    // from string to int format on ...

    // conditions
    from_string_to_int_format(string_conditions, conditions);
    string_conditions.clear();
    string_conditions.resize(0);

    // delimiters
    from_string_to_int_format(string_delimiters, delimiters);
    string_delimiters.clear();
    string_delimiters.resize(0);

    // ignored_context
    from_string_to_int_format(string_ignored_context, ignored_context);
    string_ignored_context.clear();
    string_ignored_context.resize(0);

    // gathering information from source VMat

    indices.clear();
    indices.resize(0);
    int current_context_length = 0;
    bool target_contains_missing = false;
    PP<ProgressBar> pb = new ProgressBar("Gathering information from source VMat of length " + tostring(source->length()), source->length());
    for(int i=0; i<source->length(); i++)
    {
        source->getRow(i,row);

        if(is_true(delimiters,row))
        {
            max_context_length = max_context_length < current_context_length ? current_context_length : max_context_length;
            current_context_length = 0;
        }
        else
        {
            if(!full_context || !is_true(ignored_context,row)) current_context_length++;
        }

        // Testing conditions for inclusion/exclusion

        if(i + conditions_offset >= 0 && i + conditions_offset < source->length())
        {
            source->getRow(i+conditions_offset,row);
            if(exclude_missing_target_tokens)
            {
                target_contains_missing = false;
                int ni = source->targetsize()+source->inputsize();
                for(int j=source->inputsize(); j<ni; j++)
                    if(is_missing(row[j]))
                    {
                        target_contains_missing = true;
                        break;
                    }
            }

            if(!exclude_missing_target_tokens || !target_contains_missing)
            {
                if(is_true(conditions,row))
                {
                    if(!conditions_for_exclusion) indices.append(i);
                }
                else
                {
                    if(conditions_for_exclusion) indices.append(i);
                }
            }
        }
        pb->update(i+1);
    }

    max_context_length = max_context_length < current_context_length ? current_context_length : max_context_length;

    if(n_left_context >= 0 && n_right_context >= 0)
        max_context_length = 1 + n_left_context + n_right_context;

    length_ = indices.length();
    width_ = n_attributes*(max_context_length);

    if(inputsize_ < 0) inputsize_ = max_context_length * source->inputsize();
    if(targetsize_ < 0 && put_only_target_attributes) targetsize_ = source->targetsize();
    if(targetsize_ < 0 && !put_only_target_attributes) targetsize_ = max_context_length * source->targetsize();
    if(weightsize_ < 0) weightsize_ = source->weightsize();
    if(weightsize_ > 0) PLERROR("In ProcessSymbolicSequenceVMatrix:build_() : does not support weightsize > 0");

    current_row_i.resize(n_attributes*(max_context_length));
    current_target_pos = -1;
    current_context_pos.clear();
    lower_bound = 1;
    upper_bound = -1;

    if(n_left_context < 0)
    {
        left_context.resize(width_);
        left_positions.resize(max_context_length);
    }
    else
    {
        left_context.resize(n_attributes*n_left_context);
        left_positions.resize(n_left_context);
    }

    if(n_right_context < 0)
    {
        right_context.resize(width_);
        right_positions.resize(max_context_length);
    }
    else
    {
        right_context.resize(n_attributes*n_right_context);
        right_positions.resize(n_right_context);
    }

    if(put_only_target_attributes)
    {
        width_ = source->inputsize()*max_context_length + source->targetsize();
    }

    if(inputsize_+targetsize_ != width_) PLERROR("In ProcessSymbolicSequenceVMatrix:build_() : inputsize_ + targetsize_ != width_");


    // Should we call:
    // setMetaInfoFromSource(); // ?
    updateMtime(source);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::ProcessSymbolicSequenceVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

void PLearn::ProcessSymbolicSequenceVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 142 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::OptionBase::buildoption, conditions, conditions_for_exclusion, conditions_offset, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), delimiters, exclude_missing_target_tokens, full_context, ignored_context, n_left_context, n_right_context, put_only_target_attributes, string_conditions, string_delimiters, string_ignored_context, and use_last_context.

{
    // ### Declare all of this object's options here
    // ### For the "flags" of each option, you should typically specify
    // ### one of OptionBase::buildoption, OptionBase::learntoption or
    // ### OptionBase::tuningoption. Another possible flag to be combined with
    // ### is OptionBase::nosave

    declareOption(ol, "n_left_context",
                  &ProcessSymbolicSequenceVMatrix::n_left_context,
                  OptionBase::buildoption,
                  "Number of elements at the left of (or before) the target"
                  " element.\n"
                  "(if < 0, all elements to the left are included until a"
                  " delimiter is met)\n");

    declareOption(ol, "n_right_context",
                  &ProcessSymbolicSequenceVMatrix::n_right_context,
                  OptionBase::buildoption,
                  "Number of elements at the right of (or after) the target"
                  " element.\n"
                  "(if < 0, all elements to the right are included until a"
                  " delimiter is met)\n");

    declareOption(ol, "conditions_offset",
                  &ProcessSymbolicSequenceVMatrix::conditions_offset,
                  OptionBase::buildoption,
                  "Offset for the position of the element on which conditions"
                  " are tested\n"
                  "(default = 0)\n");

    declareOption(ol, "conditions_for_exclusion",
                  &ProcessSymbolicSequenceVMatrix::conditions_for_exclusion,
                  OptionBase::buildoption,
                  "Indication that the specified conditions are for the"
                  " exclusion (true)\n"
                  "or inclusion (false) of elements in the VMatrix\n");

    declareOption(ol, "full_context",
                  &ProcessSymbolicSequenceVMatrix::full_context,
                  OptionBase::buildoption,
                  "Indication that ignored elements of context should be"
                  " replaced by the\n"
                  "next nearest valid element\n");

    declareOption(ol, "put_only_target_attributes",
                  &ProcessSymbolicSequenceVMatrix::put_only_target_attributes,
                  OptionBase::buildoption,
                  "Indication that the only target fields of the VMatrix rows"
                  " should be\n"
                  "the (target) attributes of the context's target element\n");

    declareOption(ol, "use_last_context",
                  &ProcessSymbolicSequenceVMatrix::use_last_context,
                  OptionBase::buildoption,
                  "Indication that the last accessed context should be put in"
                  " a buffer.\n");

    declareOption(ol, "exclude_missing_target_tokens",
                  &ProcessSymbolicSequenceVMatrix::exclude_missing_target_tokens,
                  OptionBase::buildoption,
                  "Indication to exclude from the VMatrix the context centered\n"
                  "around a token with missing value target fields.\n");



    declareOption(ol, "conditions",
                  &ProcessSymbolicSequenceVMatrix::conditions,
                  OptionBase::buildoption,
                  "Conditions to be satisfied for the exclusion or inclusion"
                  " (see\n"
                  "conditions_for_exclusion) of elements in the VMatrix\n");

    declareOption(ol, "string_conditions",
                  &ProcessSymbolicSequenceVMatrix::string_conditions,
                  OptionBase::buildoption,
                  "Conditions, in string format, to be satisfied for the"
                  " exclusion or\n"
                  "inclusion (see conditions_for_exclusion) of elements in the"
                  " VMatrix\n");

    declareOption(ol, "delimiters",
                  &ProcessSymbolicSequenceVMatrix::delimiters,
                  OptionBase::buildoption,
                  "Delimiters of context\n");

    declareOption(ol, "string_delimiters",
                  &ProcessSymbolicSequenceVMatrix::string_delimiters,
                  OptionBase::buildoption,
                  "Delimiters, in string format, of context\n");

    declareOption(ol, "ignored_context",
                  &ProcessSymbolicSequenceVMatrix::ignored_context,
                  OptionBase::buildoption,
                  "Elements to be ignored in context\n");

    declareOption(ol, "string_ignored_context",
                  &ProcessSymbolicSequenceVMatrix::string_ignored_context,
                  OptionBase::buildoption,
                  "Elements, in string format, to be ignored in context\n");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::ProcessSymbolicSequenceVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 309 of file ProcessSymbolicSequenceVMatrix.h.

ProcessSymbolicSequenceVMatrix * PLearn::ProcessSymbolicSequenceVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

void PLearn::ProcessSymbolicSequenceVMatrix::fill_current_row ( int  i,
int cp,
int target_position 
) const [private]

Fills the current_row_i field, and returns the number of non missing values for the ith context.

Definition at line 523 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::cp(), current_context_pos, current_row_i, current_target_pos, delimiters, element, PLearn::VMatrix::fill(), PLearn::TVec< T >::fill(), fixed_context, full_context, i, ignored_context, indices, is_true(), left_context, left_positions, PLearn::TVec< T >::length(), PLearn::VMat::length(), lower_bound, MISSING_VALUE, n_attributes, n_left_context, n_right_context, right_context, right_positions, PLearn::SourceVMatrix::source, PLearn::TVec< T >::subVec(), target_element, upper_bound, and use_last_context.

Referenced by getExample(), and getNewRow().

{

    int target = indices[i];

    // If the target element is a delimiter, do nothing: Hugo: may not be a good thing!
    source->getRow(target,target_element);
    /*
      if(is_true(delimiters,target_element))
      {
      v.fill(MISSING_VALUE);
      if(use_last_context)
      {
      current_context_pos.clear();
      current_row_i.fill(MISSING_VALUE);
      current_target_pos = -1;
      lower_bound = 1;
      upper_bound = -1;
      }
      return;
      }
    */

    int left_added_elements = 0;
    int right_added_elements = 0;

    int left_pos = 1;
    int right_pos = 1;
    int p = 0;

    bool to_add = false;

    int this_lower_bound = target;
    int this_upper_bound = target;

    // Left context construction

    while(n_left_context < 0 || left_added_elements < n_left_context)
    {
        to_add = false;


        p = target-left_pos;

        if(p < 0 || p >= source->length())
            break;

        // Verifying if context can be found in last context
        if(use_last_context && lower_bound <= p && upper_bound >= p)
        {
            if(current_context_pos.find(p) != current_context_pos.end())
            {
                int position = current_context_pos[p];
                element = current_row_i.subVec(n_attributes*position,n_attributes);
                if(!is_true(ignored_context,element))
                {
                    to_add = true;
                }
            }
            else
            {
                left_pos++;
                continue;
            }
        }  // If not, fetch element in source VMat
        else
        {
            source->getRow(p,element);
            if(!is_true(ignored_context,element))
                to_add = true;
        }

        if(is_true(delimiters,element)) break;

        if(to_add)
        {
            left_context.subVec(n_attributes*left_added_elements,n_attributes) << element;
            if(use_last_context) left_positions[left_added_elements] = p;
            this_lower_bound = p;
            left_added_elements++;
        }
        else
            if(!full_context)
            {
                left_context.subVec(n_attributes*left_added_elements,n_attributes).fill(MISSING_VALUE);
                if(use_last_context) left_positions[left_added_elements] = p;
                this_lower_bound = p;
                left_added_elements++;
            }

        left_pos++;
    }

    // Right context construction

    while(n_right_context < 0 || right_added_elements < n_right_context)
    {
        to_add = false;

        p = target+right_pos;

        if(p < 0 || p >= source->length())
            break;

        // Verifying if context can be found in last context
        if(use_last_context && lower_bound <= p && upper_bound >= p)
        {
            if(current_context_pos.find(p) != current_context_pos.end())
            {
                int position = current_context_pos[p];
                element = current_row_i.subVec(n_attributes*position,n_attributes);
                if(!is_true(ignored_context,element))
                {
                    to_add = true;
                }
            }
            else
            {
                right_pos++;
                continue;
            }
        }  // If not, fetch element in source VMat
        else
        {
            source->getRow(p,element);
            if(!is_true(ignored_context,element))
                to_add = true;
        }

        if(is_true(delimiters,element)) break;

        if(to_add)
        {
            right_context.subVec(n_attributes*right_added_elements,n_attributes) << element;
            if(use_last_context) right_positions[right_added_elements] = p;
            this_upper_bound = p;
            right_added_elements++;
        }
        else
            if(!full_context)
            {
                right_context.subVec(n_attributes*right_added_elements,n_attributes).fill(MISSING_VALUE);
                if(use_last_context) right_positions[right_added_elements] = p;
                this_upper_bound = p;
                right_added_elements++;
            }

        right_pos++;
    }

    current_context_pos.clear();
    current_target_pos = -1;
    lower_bound = this_lower_bound;
    upper_bound = this_upper_bound;
    //current_row_i.fill(MISSING_VALUES);

    // Constructing complete row

    cp = 0;

    if(fixed_context)  // Adding missing value, for the case where the context is of fixed length
        if(n_left_context-left_added_elements>0)
        {
            current_row_i.subVec(cp*n_attributes,n_attributes*(n_left_context-left_added_elements)).fill(MISSING_VALUE);
            cp = n_left_context-left_added_elements;
        }

    // adding left context

    int temp = left_added_elements;

    while(temp > 0)
    {
        temp--;
        current_row_i.subVec(cp*n_attributes,n_attributes) << left_context.subVec(temp*n_attributes,n_attributes);
        if(use_last_context) current_context_pos[(int)left_positions[temp]] = cp;
        cp++;
    }

    // adding target element
    target_position = cp;
    current_row_i.subVec(cp*n_attributes,n_attributes) << target_element;
    if(use_last_context)
    {
        current_context_pos[target] = cp;
        current_target_pos = cp;
    }
    cp++;

    // adding right context

    int r=0;
    while(r<right_added_elements)
    {
        current_row_i.subVec(cp*n_attributes,n_attributes) << right_context.subVec(r*n_attributes,n_attributes);
        if(use_last_context) current_context_pos[(int)right_positions[r]] = cp;
        r++;
        cp++;
    }

    if(fixed_context && current_row_i.length()-cp*n_attributes > 0)
    {
        current_row_i.subVec(cp*n_attributes,current_row_i.length()-cp*n_attributes).fill(MISSING_VALUE);
        cp = current_row_i.length()/n_attributes;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::from_string_to_int_format ( TVec< TVec< pair< int, string > > > &  str_format,
TVec< TVec< pair< int, int > > > &  int_format 
) [inline, private]

Definition at line 215 of file ProcessSymbolicSequenceVMatrix.h.

References i.

Referenced by build_().

    {
        int int_format_length = int_format.length();
        int_format.resize(int_format_length + str_format.length());
        for(int i=0; i<str_format.length(); i++)
            from_string_to_int_format(str_format[i],
                                      int_format[i+int_format_length]);
    }

Here is the caller graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::from_string_to_int_format ( TVec< pair< int, string > > &  str_format,
TVec< pair< int, int > > &  int_format 
) [inline, private]

Definition at line 225 of file ProcessSymbolicSequenceVMatrix.h.

References i, PLearn::is_missing(), and PLERROR.

    {
        int int_format_length = int_format.length();
        int_format.resize(int_format_length+str_format.length());
        for(int i=0; i<str_format.length(); i++)
        {
            int_format[int_format_length+i].first = str_format[i].first;
            real value = source->getStringVal(str_format[i].first,
                                              str_format[i].second);
            if(is_missing(value))
                PLERROR("In ProcessSymbolicSequenceVMatrix::"
                        "from_string_to_int_format :\n"
                        "%s not a valid string symbol for column %d\n",
                        str_format[i].second.c_str(), str_format[i].first);
            int_format[int_format_length+i].second = (int)value;
        }
    }

Here is the call graph for this function:

PP< Dictionary > PLearn::ProcessSymbolicSequenceVMatrix::getDictionary ( int  col) const [virtual]

Return the Dictionary object for a certain field, or a null pointer if there isn't one.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 486 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::VMatrix::inputsize(), max_context_length, PLERROR, PLearn::SourceVMatrix::source, and PLearn::VMatrix::width_.

{
    if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getDictionary() : invalid col %d, width()=%d", col, width_);
    if(source)
    {
        int src_col;
        if(col < max_context_length * source->inputsize())
            src_col = col%source->inputsize();
        else
            src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize();
        return source->getDictionary(src_col);
    }
    return 0;
}

Here is the call graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::getExample ( int  i,
Vec input,
Vec target,
real weight 
) [virtual]

Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.

If not a weighted matrix, weight should be set to default value 1.

Reimplemented from PLearn::VMatrix.

Definition at line 731 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::cp(), current_row_i, fill_current_row(), fixed_context, PLearn::VMatrix::inputsize(), PLearn::VMatrix::inputsize_, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, n_attributes, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, PLearn::VMatrix::targetsize(), PLearn::VMatrix::targetsize_, PLearn::VMatrix::weightsize_, and PLearn::VMatrix::width_.

{
    if(i<0 || i>=length_) PLERROR("In SelectAttributeSequenceVMatrix::getExample() : invalid row acces i=%d for VMatrix(%d,%d)",i,length_,width_);

    int target_position,cp;
    fill_current_row(i,cp,target_position);

    // Seperating input and output fields
    input.resize(cp*source->inputsize());
    target.resize(put_only_target_attributes ? source->targetsize() : cp*source->targetsize());
    if(weightsize_ == 0)
        weight = 1;
    else
        PLERROR("In ProcessSymbolicSequenceVMatrix::getExample(): weighsize() > 0 not implemented");
    for(int t=0; t<cp*n_attributes; t++)
    {
        if(t%n_attributes < source->inputsize())
            input[t/n_attributes * source->inputsize() + t%n_attributes] = current_row_i[t];
        else
            if(put_only_target_attributes)
            {
                if(t/n_attributes == target_position)
                    target[t%n_attributes - source->inputsize() ] = current_row_i[t];
            }
            else
                target[t/n_attributes * source->targetsize() + t%n_attributes - source->inputsize() ] = current_row_i[t];
    }

    if(fixed_context)
    {
        // Verify if inputsize and targetsize were redefined
        if(inputsize_ > input.length())
        {
            input.resize(inputsize_);
            int it = 0;
            for(int t=cp*source->inputsize(); t<inputsize_; t++)
                input[t] = target[it++];
            for(int t=0; t<targetsize_; t++)
                target[t] = target[it++];
            target.resize(targetsize_);
        }
        else if(targetsize() > target.length())
        {
            target.resize(targetsize_);
            int it = cp*source->inputsize()-inputsize_;
            for(int t=0; it<targetsize_; t++)
                target[it++] = target[t];
            it = inputsize_;
            for(int t=0; it<input.length(); t++)
                target[t] = input[it++];
            input.resize(inputsize_);
        }
    }

    return;
}

Here is the call graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

Fill the vector 'v' with the content of the i-th row.

v is assumed to be the right size.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 109 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::cp(), current_row_i, PLearn::VMatrix::fill(), fill_current_row(), PLearn::VMatrix::inputsize(), PLearn::TVec< T >::length(), PLearn::VMatrix::length_, max_context_length, MISSING_VALUE, n_attributes, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, PLearn::TVec< T >::subVec(), and PLearn::VMatrix::width_.

{
    if(i<0 || i>=length_)
        PLERROR("In SelectAttributeSequenceVMatrix::getNewRow() :\n"
                " invalid row acces i=%d for VMatrix(%d,%d)\n",
                i,length_,width_);

    current_row_i.resize(n_attributes*(max_context_length));
    int cp,target_position;
    fill_current_row(i,cp,target_position);

    if(current_row_i.length()-cp*n_attributes > 0)
        current_row_i.subVec(cp*n_attributes,current_row_i.length()-cp*n_attributes).fill(MISSING_VALUE);

    // Seperating input and output fields

    for(int t=0; t<current_row_i.length(); t++)
    {
        if(t%n_attributes < source->inputsize())
            v[t/n_attributes * source->inputsize() + t%n_attributes] = current_row_i[t];
        else
            if(put_only_target_attributes)
            {
                if(t/n_attributes == target_position)
                    v[max_context_length*source->inputsize() + t%n_attributes - source->inputsize() ] = current_row_i[t];
            }
            else
                v[max_context_length*source->inputsize() + t/n_attributes * source->targetsize() + t%n_attributes - source->inputsize() ] = current_row_i[t];
    }

    return;
}

Here is the call graph for this function:

OptionList & PLearn::ProcessSymbolicSequenceVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

OptionMap & PLearn::ProcessSymbolicSequenceVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

RemoteMethodMap & PLearn::ProcessSymbolicSequenceVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.

real PLearn::ProcessSymbolicSequenceVMatrix::getStringVal ( int  col,
const string &  str 
) const [virtual]

returns value associated with a string (or MISSING_VALUE if there's no association for this string)

Reimplemented from PLearn::VMatrix.

Definition at line 394 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::VMatrix::inputsize(), max_context_length, MISSING_VALUE, and PLearn::SourceVMatrix::source.

{
    if(source)
    {
        int src_col;
        if(col < max_context_length * source->inputsize())
            src_col = col%source->inputsize();
        else
            src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize();
        return source->getStringVal(src_col,str);
    }

    return MISSING_VALUE;
}

Here is the call graph for this function:

string PLearn::ProcessSymbolicSequenceVMatrix::getValString ( int  col,
real  val 
) const [virtual]

returns the string associated with value val for field# col.

Or returns "" if no string is associated.

Reimplemented from PLearn::VMatrix.

Definition at line 409 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::VMatrix::inputsize(), max_context_length, and PLearn::SourceVMatrix::source.

{
    if(source)
    {
        int src_col;
        if(col < max_context_length * source->inputsize())
            src_col = col%source->inputsize();
        else
            src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize();
        return source->getValString(src_col,val);
    }

    return "";
}

Here is the call graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::getValues ( int  row,
int  col,
Vec values 
) const [virtual]

Gives the possible values for a certain field in the VMatrix.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 424 of file ProcessSymbolicSequenceVMatrix.cc.

References indices, PLearn::VMatrix::inputsize(), PLearn::VMatrix::length_, max_context_length, PLERROR, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, and PLearn::VMatrix::width_.

{
    if(row < 0 || row >= length_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid row %d, length()=%d", row, length_);
    if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid col %d, width()=%d", col, width_);
    if(source)
    {
        int src_col;
        if(col < max_context_length * source->inputsize())
            src_col = col%source->inputsize();
        else
            src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize();
        source->getValues(indices[row],src_col, values);
    }
    else
        values.resize(0);
}

Here is the call graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::getValues ( const Vec input,
int  col,
Vec values 
) const [virtual]

Gives the possible values of a certain field (column) given the input.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 441 of file ProcessSymbolicSequenceVMatrix.cc.

References PLearn::TVec< T >::fill(), PLearn::VMatrix::inputsize_, max_context_length, MISSING_VALUE, n_left_context, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, subinput, PLearn::TVec< T >::subVec(), and PLearn::VMatrix::width_.

{
    if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid col %d, width()=%d", col, width_);
    if(source)
    {
        int sinputsize = source->inputsize();
        int stargetsize = source->targetsize();
        int src_col,this_col;
        if(col < max_context_length * sinputsize)
            src_col = col%sinputsize;
        else
            src_col = sinputsize + (col-max_context_length * sinputsize)%stargetsize;

        // Fill subinput input part
        subinput.resize(sinputsize);
        if(col>=inputsize_)
        {
            if(put_only_target_attributes)
                this_col = n_left_context;
            else
                this_col = (col-inputsize_)/stargetsize;
        }
        else
            this_col = col/sinputsize;
        subinput << input.subVec(this_col*sinputsize,sinputsize);

        // Fill subinput target part
        subinput.resize(sinputsize+stargetsize);
        if(!put_only_target_attributes || col == inputsize_ || (col<inputsize_ && col/sinputsize == n_left_context))
        {
            if(put_only_target_attributes)
                subinput.subVec(sinputsize,stargetsize) << input.subVec(inputsize_,stargetsize);
            else
                subinput.subVec(sinputsize,stargetsize) << input.subVec(inputsize_+this_col*stargetsize,stargetsize);
                
        }
        else
        {
            subinput.subVec(sinputsize,stargetsize).fill(MISSING_VALUE);
        }
        source->getValues(subinput,src_col,values);
    }
    else values.resize(0);
}

Here is the call graph for this function:

bool PLearn::ProcessSymbolicSequenceVMatrix::is_true ( const TVec< TVec< pair< int, int > > > &  conditions,
const Vec  row 
) const [inline, private]

Definition at line 244 of file ProcessSymbolicSequenceVMatrix.h.

References i, j, and PLERROR.

Referenced by build_(), and fill_current_row().

    {
        bool condition_satisfied;
        for(int i=0; i<conditions.length(); i++)
        {
            condition_satisfied = true;
            if(conditions[i].length() == 0)
                PLERROR("ProcessSymbolicSequenceVMatrix::is_true :\n"
                        "conditions[%d] should not be empty\n", i);
            for(int j=0; j<conditions[i].length(); j++)
            {
                if(row[conditions[i][j].first] != conditions[i][j].second)
                {
                    condition_satisfied = false;
                    break;
                }
            }
            if(condition_satisfied) return true;
        }

        return false;
    }

Here is the caller graph for this function:

void PLearn::ProcessSymbolicSequenceVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Member Data Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 309 of file ProcessSymbolicSequenceVMatrix.h.

Conditions to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.

Definition at line 171 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Indication that the specified conditions are for the exclusion (true) or inclusion (false) of elements in the VMatrix.

Definition at line 152 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and declareOptions().

Offset for the position of the element on which conditions are tested (default = 0).

Definition at line 148 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and declareOptions().

Mapping from the position in the source sequence to the position in the last context.

Definition at line 114 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and fill_current_row().

Position in the last context of the target element (symbol and attributes)

Definition at line 104 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and fill_current_row().

Delimiters of context.

Definition at line 178 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), fill_current_row(), and makeDeepCopyFromShallowCopy().

Indication to exclude from the VMatrix the context centered around a token with missing value target fields.

Definition at line 167 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and declareOptions().

Indication that the context is of fixed length.

Definition at line 94 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), fill_current_row(), and getExample().

Indication that ignored elements of context should be replaced by the next nearest valid element.

Definition at line 156 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), and fill_current_row().

Elements to be ignored in context.

Definition at line 184 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), fill_current_row(), and makeDeepCopyFromShallowCopy().

Indices of the selected contexts.

Definition at line 117 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), fill_current_row(), getValues(), and makeDeepCopyFromShallowCopy().

Temporary fields.

Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().

Temporary fields.

Definition at line 120 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().

Position in the source sequence of the first element in the last context.

Definition at line 107 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and fill_current_row().

Maximum length of a context.

Definition at line 96 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), getDictionary(), getNewRow(), getStringVal(), getValString(), and getValues().

Number of attributes.

Definition at line 98 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), fill_current_row(), getExample(), and getNewRow().

Number of elements at the left of the target element.

If < 0, all elements to the left are included until a delimiter is met.

Definition at line 140 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), fill_current_row(), getValues(), and ProcessSymbolicSequenceVMatrix().

Number of elements at the right of the target element.

If < 0, all elements to the right are included until a delimiter is met

Definition at line 144 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), fill_current_row(), and ProcessSymbolicSequenceVMatrix().

Indication that the only target fields of the VMatrix rows should be the (target) attributes of the context's target element.

Definition at line 160 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), getExample(), getNewRow(), and getValues().

Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and makeDeepCopyFromShallowCopy().

Conditions, in string format, to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.

Definition at line 175 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Delimiters, in string format, of context.

Definition at line 181 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Elements, in string format, to be ignored in context.

Definition at line 187 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by getValues(), and makeDeepCopyFromShallowCopy().

Position in the source sequence of the last element in the last context.

Definition at line 109 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by build_(), and fill_current_row().

Indication that the last accessed context should be put in a buffer.

Definition at line 163 of file ProcessSymbolicSequenceVMatrix.h.

Referenced by declareOptions(), and fill_current_row().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines