PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions | Private Attributes
PLearn::HashMapFeatureSet Class Reference

Base class for feature sets that maintains an explicit mapping between index and string form features", This class facilitates the conception of FeatureSet objects. More...

#include <HashMapFeatureSet.h>

Inheritance diagram for PLearn::HashMapFeatureSet:
Inheritance graph
[legend]
Collaboration diagram for PLearn::HashMapFeatureSet:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 HashMapFeatureSet ()
 Default constructor.
virtual HashMapFeatureSetdeepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual void getFeatures (string token, TVec< int > &feats)
 Gives features of token in index form.
virtual string getStringFeature (int index)
 Gives string form of a feature in index form.
virtual int getIndexFeature (string str)
 Gives index form of a feature in string form.
virtual int size ()
 Gives the number of features in the set.
virtual void addFeatures (string token)
 Adds the features for a given token in the set.
virtual void addFeatures (VMat tokens, int min_freq=-1)
 Adds the features for the tokens contained in the set.
virtual void clear ()
 Clears all features from the feature set.
virtual void getNewFeaturesString (string token, TVec< string > &feats_str)=0
 Gives the possibly new features in string form for a token.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

bool report_progress
 Indication that progress should be reported in a ProgressBar when filling the feature set.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

hash_map< int, string > index_to_string_feats
 Index feature to string feature mapping.
hash_map< string, intstring_to_index_feats
 String feature to index feature mapping.

Private Types

typedef FeatureSet inherited

Private Member Functions

void build_ ()
 This does the actual building.

Private Attributes

TVec< string > f_str
 Temporary computations vectors.
int index

Detailed Description

Base class for feature sets that maintains an explicit mapping between index and string form features", This class facilitates the conception of FeatureSet objects.


" Classes that inherits from this class only need to define the\n" getNewFeaturesString() function, which is called when adding new features
" or when asking for the features of a token (in index or string format).\n" This function should return the list of features in string format
" for a given token. It is assumed that no feature is present more than once\n" in the output of that function, for any input token.
" This base class maintains two hash_maps between the index and string forms\n" of the features. These mappings contain all the necessary
" information to define the different functions of a FeatureSet object.\n"

Definition at line 61 of file HashMapFeatureSet.h.


Member Typedef Documentation

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 63 of file HashMapFeatureSet.h.


Constructor & Destructor Documentation

PLearn::HashMapFeatureSet::HashMapFeatureSet ( )

Default constructor.

Definition at line 61 of file HashMapFeatureSet.cc.

    : report_progress(false)
{}

Member Function Documentation

string PLearn::HashMapFeatureSet::_classname_ ( ) [static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

OptionList & PLearn::HashMapFeatureSet::_getOptionList_ ( ) [static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

RemoteMethodMap & PLearn::HashMapFeatureSet::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

bool PLearn::HashMapFeatureSet::_isa_ ( const Object o) [static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

StaticInitializer HashMapFeatureSet::_static_initializer_ & PLearn::HashMapFeatureSet::_static_initialize_ ( ) [static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

void PLearn::HashMapFeatureSet::addFeatures ( string  token) [virtual]

Adds the features for a given token in the set.

Implements PLearn::FeatureSet.

Definition at line 142 of file HashMapFeatureSet.cc.

References f_str, getNewFeaturesString(), index, index_to_string_feats, PLearn::TVec< T >::length(), and string_to_index_feats.

{
    string str;
    int index;
    
    getNewFeaturesString(token,f_str);
    for(int t=0; t<f_str.length(); t++)
    {
        str = f_str[t];
        if(string_to_index_feats.find(str) == string_to_index_feats.end()){
            index = int(string_to_index_feats.size());
            string_to_index_feats[str] = index;
            index_to_string_feats[index] = str;
        }
    }

}

Here is the call graph for this function:

void PLearn::HashMapFeatureSet::addFeatures ( VMat  tokens,
int  min_freq = -1 
) [virtual]

Adds the features for the tokens contained in the set.

Features with frequency smaller than min_freq are not added.

Implements PLearn::FeatureSet.

Definition at line 160 of file HashMapFeatureSet.cc.

References PLearn::TVec< T >::copy(), f_str, getNewFeaturesString(), i, index, index_to_string_feats, j, PLearn::TVec< T >::length(), PLearn::VMat::length(), report_progress, PLearn::TVec< T >::resize(), string_to_index_feats, PLearn::tostring(), and PLearn::VMat::width().

{
    string str, token;
    int index;
    map<string,int> frequencies;
    map<string,TVec<string> > token_features;
    map<string,bool> token_feat_inserted;
    Vec row(tokens->width());
    TVec<string> f_str_temp;

    PP<ProgressBar> pb;
    int id=0;

    if(min_freq > 1)
    {
        if(report_progress)
            pb = new ProgressBar("Counting frequencies for the features of " + tostring(tokens->length()*tokens->width()) + " tokens", tokens->length()*tokens->width());
        
        for(int i=0; i<tokens->length(); i++)
        {
            tokens->getRow(i,row);
            for(int j=0; j<tokens->width(); j++)
            {
                token = tokens->getValString(j,row[j]);
                if(token_features.find(token) == token_features.end())
                {
                    getNewFeaturesString(token,f_str);
                    token_features[token] = f_str.copy();
                }
                else
                {
                    f_str_temp = token_features[token];
                    f_str.resize(f_str_temp.length());
                    f_str << f_str_temp;
                }

                for(int t=0; t<f_str.length(); t++)
                {
                    str = f_str[t];
                    if(frequencies.find(str) == frequencies.end())
                        frequencies[str] = 1;
                    else
                        frequencies[str]++;
                }
                
                if(report_progress) pb->update(++id);
            }        
        }
        token_features.clear();
    }

    id=0;
    if(report_progress)
        pb = new ProgressBar("Filling FeatureSet with the features of " + tostring(tokens->length()*tokens->width()) + " tokens", tokens->length()*tokens->width());


    for(int i=0; i<tokens->length(); i++)
    {
        tokens->getRow(i,row);
        for(int j=0; j<tokens->width(); j++)
        {
            token = tokens->getValString(j,row[j]);
            if(token_feat_inserted.find(token) == token_feat_inserted.end())
            {
                getNewFeaturesString(token,f_str);
                for(int t=0; t<f_str.length(); t++)
                {
                    str = f_str[t];
                    if(string_to_index_feats.find(str) == string_to_index_feats.end()
                       && (min_freq <=1 || frequencies[str] >= min_freq)){
                        index = int(string_to_index_feats.size());
                        string_to_index_feats[str] = index;
                        index_to_string_feats[index] = str;
                    }
                }
                token_feat_inserted[token] = true;
            }
            if(report_progress) pb->update(++id);
        }
    }
}

Here is the call graph for this function:

void PLearn::HashMapFeatureSet::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 66 of file HashMapFeatureSet.cc.

References PLearn::FeatureSet::build(), and build_().

Referenced by PLearn::WordNetFeatureSet::build(), and PLearn::PythonFeatureSet::build().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HashMapFeatureSet::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 98 of file HashMapFeatureSet.cc.

Referenced by build().

{}

Here is the caller graph for this function:

void PLearn::HashMapFeatureSet::clear ( ) [virtual]

Clears all features from the feature set.

Implements PLearn::FeatureSet.

Definition at line 242 of file HashMapFeatureSet.cc.

References index_to_string_feats, and string_to_index_feats.

void PLearn::HashMapFeatureSet::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 81 of file HashMapFeatureSet.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::FeatureSet::declareOptions(), index_to_string_feats, PLearn::OptionBase::learntoption, report_progress, and string_to_index_feats.

Referenced by PLearn::WordNetFeatureSet::declareOptions(), and PLearn::PythonFeatureSet::declareOptions().

{
    declareOption(ol, "report_progress", &HashMapFeatureSet::report_progress,
                  OptionBase::buildoption,
                  "Indication that progress should be reported in a ProgressBar\n"
                  "when filling the feature set.\n");    
    declareOption(ol, "index_to_string_feats", &HashMapFeatureSet::index_to_string_feats,
                  OptionBase::learntoption,
                  "Index feature to string feature mapping");    
    declareOption(ol, "string_to_index_feats", &HashMapFeatureSet::string_to_index_feats,
                  OptionBase::learntoption,
                  "String feature to index feature mapping");    

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

Here is the caller graph for this function:

static const PPath& PLearn::HashMapFeatureSet::declaringFile ( ) [inline, static]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 81 of file HashMapFeatureSet.h.

:
    //#####  Protected Options  ###############################################
HashMapFeatureSet * PLearn::HashMapFeatureSet::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 59 of file HashMapFeatureSet.cc.

void PLearn::HashMapFeatureSet::getFeatures ( string  token,
TVec< int > &  feats 
) [virtual]

Gives features of token in index form.

Implements PLearn::FeatureSet.

Definition at line 101 of file HashMapFeatureSet.cc.

References f_str, getNewFeaturesString(), index, PLearn::TVec< T >::length(), PLearn::TVec< T >::push_back(), PLearn::TVec< T >::resize(), and string_to_index_feats.

{
    int index;
    string str;
    getNewFeaturesString(token,f_str);
    feats.resize(0);
    
    for(int t=0; t<f_str.length(); t++)
    {
        str = f_str[t];
        if(string_to_index_feats.find(str) != string_to_index_feats.end())
        {
            index = string_to_index_feats[str];
            feats.push_back(index);
        }
    }
}

Here is the call graph for this function:

int PLearn::HashMapFeatureSet::getIndexFeature ( string  str) [virtual]

Gives index form of a feature in string form.

Implements PLearn::FeatureSet.

Definition at line 129 of file HashMapFeatureSet.cc.

References PLERROR, and string_to_index_feats.

{
    if(string_to_index_feats.find(str) == string_to_index_feats.end())
        PLERROR("In HashMapFeatureSet::getIndexFeature(): string %s is an invalid feature", str.c_str());

    return string_to_index_feats[str];
}
virtual void PLearn::HashMapFeatureSet::getNewFeaturesString ( string  token,
TVec< string > &  feats_str 
) [pure virtual]

Gives the possibly new features in string form for a token.

Implements PLearn::FeatureSet.

Implemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Referenced by addFeatures(), and getFeatures().

Here is the caller graph for this function:

string PLearn::HashMapFeatureSet::getStringFeature ( int  index) [virtual]

Gives string form of a feature in index form.

Implements PLearn::FeatureSet.

Definition at line 119 of file HashMapFeatureSet.cc.

References index, index_to_string_feats, and PLERROR.

{
#ifdef BOUNDCHECK
    if(index_to_string_feats.find(index) == index_to_string_feats.end())
        PLERROR("In HashMapFeatureSet::getStringFeature(): index %d is an invalid feature index", index);
#endif
    return index_to_string_feats[index];
    
}
void PLearn::HashMapFeatureSet::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 72 of file HashMapFeatureSet.cc.

References PLearn::deepCopyField(), index_to_string_feats, PLearn::FeatureSet::makeDeepCopyFromShallowCopy(), and string_to_index_feats.

Referenced by PLearn::WordNetFeatureSet::makeDeepCopyFromShallowCopy(), and PLearn::PythonFeatureSet::makeDeepCopyFromShallowCopy().

{
    inherited::makeDeepCopyFromShallowCopy(copies);
    deepCopyField(index_to_string_feats, copies);
    deepCopyField(string_to_index_feats, copies);

    //PLERROR("HashMapFeatureSet::makeDeepCopyFromShallowCopy not fully (correctly) implemented yet!");
}

Here is the call graph for this function:

Here is the caller graph for this function:

int PLearn::HashMapFeatureSet::size ( ) [virtual]

Gives the number of features in the set.

Implements PLearn::FeatureSet.

Definition at line 137 of file HashMapFeatureSet.cc.

References index_to_string_feats.

{
    return int(index_to_string_feats.size());
}

Member Data Documentation

Reimplemented from PLearn::FeatureSet.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 81 of file HashMapFeatureSet.h.

Temporary computations vectors.

Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.

Definition at line 138 of file HashMapFeatureSet.h.

Referenced by addFeatures(), and getFeatures().

Definition at line 139 of file HashMapFeatureSet.h.

Referenced by addFeatures(), getFeatures(), and getStringFeature().

Index feature to string feature mapping.

Definition at line 118 of file HashMapFeatureSet.h.

Referenced by addFeatures(), clear(), declareOptions(), getStringFeature(), makeDeepCopyFromShallowCopy(), and size().

Indication that progress should be reported in a ProgressBar when filling the feature set.

Definition at line 70 of file HashMapFeatureSet.h.

Referenced by addFeatures(), and declareOptions().

String feature to index feature mapping.

Definition at line 120 of file HashMapFeatureSet.h.

Referenced by addFeatures(), clear(), declareOptions(), getFeatures(), getIndexFeature(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines