|
PLearn 0.1
|
Base class for feature sets that maintains an explicit mapping between index and string form features", This class facilitates the conception of FeatureSet objects. More...
#include <HashMapFeatureSet.h>


Public Member Functions | |
| HashMapFeatureSet () | |
| Default constructor. | |
| virtual HashMapFeatureSet * | deepCopy (CopiesMap &copies) const |
| virtual void | build () |
| Post-constructor. | |
| virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
| Transforms a shallow copy into a deep copy. | |
| virtual void | getFeatures (string token, TVec< int > &feats) |
| Gives features of token in index form. | |
| virtual string | getStringFeature (int index) |
| Gives string form of a feature in index form. | |
| virtual int | getIndexFeature (string str) |
| Gives index form of a feature in string form. | |
| virtual int | size () |
| Gives the number of features in the set. | |
| virtual void | addFeatures (string token) |
| Adds the features for a given token in the set. | |
| virtual void | addFeatures (VMat tokens, int min_freq=-1) |
| Adds the features for the tokens contained in the set. | |
| virtual void | clear () |
| Clears all features from the feature set. | |
| virtual void | getNewFeaturesString (string token, TVec< string > &feats_str)=0 |
| Gives the possibly new features in string form for a token. | |
Static Public Member Functions | |
| static string | _classname_ () |
| static OptionList & | _getOptionList_ () |
| static RemoteMethodMap & | _getRemoteMethodMap_ () |
| static bool | _isa_ (const Object *o) |
| static void | _static_initialize_ () |
| static const PPath & | declaringFile () |
Public Attributes | |
| bool | report_progress |
| Indication that progress should be reported in a ProgressBar when filling the feature set. | |
Static Public Attributes | |
| static StaticInitializer | _static_initializer_ |
Static Protected Member Functions | |
| static void | declareOptions (OptionList &ol) |
| Declares the class options. | |
Protected Attributes | |
| hash_map< int, string > | index_to_string_feats |
| Index feature to string feature mapping. | |
| hash_map< string, int > | string_to_index_feats |
| String feature to index feature mapping. | |
Private Types | |
| typedef FeatureSet | inherited |
Private Member Functions | |
| void | build_ () |
| This does the actual building. | |
Private Attributes | |
| TVec< string > | f_str |
| Temporary computations vectors. | |
| int | index |
Base class for feature sets that maintains an explicit mapping between index and string form features", This class facilitates the conception of FeatureSet objects.
"
Classes that inherits from this class only need to define the\n" getNewFeaturesString() function, which is called when adding new features
"
or when asking for the features of a token (in index or string format).\n" This function should return the list of features in string format
"
for a given token. It is assumed that no feature is present more than once\n" in the output of that function, for any input token.
"
This base class maintains two hash_maps between the index and string forms\n" of the features. These mappings contain all the necessary
"
information to define the different functions of a FeatureSet object.\n"
Definition at line 61 of file HashMapFeatureSet.h.
typedef FeatureSet PLearn::HashMapFeatureSet::inherited [private] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 63 of file HashMapFeatureSet.h.
| PLearn::HashMapFeatureSet::HashMapFeatureSet | ( | ) |
Default constructor.
Definition at line 61 of file HashMapFeatureSet.cc.
: report_progress(false) {}
| string PLearn::HashMapFeatureSet::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
| OptionList & PLearn::HashMapFeatureSet::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
| RemoteMethodMap & PLearn::HashMapFeatureSet::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
| StaticInitializer HashMapFeatureSet::_static_initializer_ & PLearn::HashMapFeatureSet::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
| void PLearn::HashMapFeatureSet::addFeatures | ( | string | token | ) | [virtual] |
Adds the features for a given token in the set.
Implements PLearn::FeatureSet.
Definition at line 142 of file HashMapFeatureSet.cc.
References f_str, getNewFeaturesString(), index, index_to_string_feats, PLearn::TVec< T >::length(), and string_to_index_feats.
{
string str;
int index;
getNewFeaturesString(token,f_str);
for(int t=0; t<f_str.length(); t++)
{
str = f_str[t];
if(string_to_index_feats.find(str) == string_to_index_feats.end()){
index = int(string_to_index_feats.size());
string_to_index_feats[str] = index;
index_to_string_feats[index] = str;
}
}
}

Adds the features for the tokens contained in the set.
Features with frequency smaller than min_freq are not added.
Implements PLearn::FeatureSet.
Definition at line 160 of file HashMapFeatureSet.cc.
References PLearn::TVec< T >::copy(), f_str, getNewFeaturesString(), i, index, index_to_string_feats, j, PLearn::TVec< T >::length(), PLearn::VMat::length(), report_progress, PLearn::TVec< T >::resize(), string_to_index_feats, PLearn::tostring(), and PLearn::VMat::width().
{
string str, token;
int index;
map<string,int> frequencies;
map<string,TVec<string> > token_features;
map<string,bool> token_feat_inserted;
Vec row(tokens->width());
TVec<string> f_str_temp;
PP<ProgressBar> pb;
int id=0;
if(min_freq > 1)
{
if(report_progress)
pb = new ProgressBar("Counting frequencies for the features of " + tostring(tokens->length()*tokens->width()) + " tokens", tokens->length()*tokens->width());
for(int i=0; i<tokens->length(); i++)
{
tokens->getRow(i,row);
for(int j=0; j<tokens->width(); j++)
{
token = tokens->getValString(j,row[j]);
if(token_features.find(token) == token_features.end())
{
getNewFeaturesString(token,f_str);
token_features[token] = f_str.copy();
}
else
{
f_str_temp = token_features[token];
f_str.resize(f_str_temp.length());
f_str << f_str_temp;
}
for(int t=0; t<f_str.length(); t++)
{
str = f_str[t];
if(frequencies.find(str) == frequencies.end())
frequencies[str] = 1;
else
frequencies[str]++;
}
if(report_progress) pb->update(++id);
}
}
token_features.clear();
}
id=0;
if(report_progress)
pb = new ProgressBar("Filling FeatureSet with the features of " + tostring(tokens->length()*tokens->width()) + " tokens", tokens->length()*tokens->width());
for(int i=0; i<tokens->length(); i++)
{
tokens->getRow(i,row);
for(int j=0; j<tokens->width(); j++)
{
token = tokens->getValString(j,row[j]);
if(token_feat_inserted.find(token) == token_feat_inserted.end())
{
getNewFeaturesString(token,f_str);
for(int t=0; t<f_str.length(); t++)
{
str = f_str[t];
if(string_to_index_feats.find(str) == string_to_index_feats.end()
&& (min_freq <=1 || frequencies[str] >= min_freq)){
index = int(string_to_index_feats.size());
string_to_index_feats[str] = index;
index_to_string_feats[index] = str;
}
}
token_feat_inserted[token] = true;
}
if(report_progress) pb->update(++id);
}
}
}

| void PLearn::HashMapFeatureSet::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 66 of file HashMapFeatureSet.cc.
References PLearn::FeatureSet::build(), and build_().
Referenced by PLearn::WordNetFeatureSet::build(), and PLearn::PythonFeatureSet::build().
{
inherited::build();
build_();
}


| void PLearn::HashMapFeatureSet::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 98 of file HashMapFeatureSet.cc.
Referenced by build().
{}

| void PLearn::HashMapFeatureSet::clear | ( | ) | [virtual] |
Clears all features from the feature set.
Implements PLearn::FeatureSet.
Definition at line 242 of file HashMapFeatureSet.cc.
References index_to_string_feats, and string_to_index_feats.
{
index_to_string_feats.clear();
string_to_index_feats.clear();
}
| void PLearn::HashMapFeatureSet::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 81 of file HashMapFeatureSet.cc.
References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::FeatureSet::declareOptions(), index_to_string_feats, PLearn::OptionBase::learntoption, report_progress, and string_to_index_feats.
Referenced by PLearn::WordNetFeatureSet::declareOptions(), and PLearn::PythonFeatureSet::declareOptions().
{
declareOption(ol, "report_progress", &HashMapFeatureSet::report_progress,
OptionBase::buildoption,
"Indication that progress should be reported in a ProgressBar\n"
"when filling the feature set.\n");
declareOption(ol, "index_to_string_feats", &HashMapFeatureSet::index_to_string_feats,
OptionBase::learntoption,
"Index feature to string feature mapping");
declareOption(ol, "string_to_index_feats", &HashMapFeatureSet::string_to_index_feats,
OptionBase::learntoption,
"String feature to index feature mapping");
// Now call the parent class' declareOptions
inherited::declareOptions(ol);
}


| static const PPath& PLearn::HashMapFeatureSet::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 81 of file HashMapFeatureSet.h.
:
//##### Protected Options ###############################################
| HashMapFeatureSet * PLearn::HashMapFeatureSet::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 59 of file HashMapFeatureSet.cc.
Gives features of token in index form.
Implements PLearn::FeatureSet.
Definition at line 101 of file HashMapFeatureSet.cc.
References f_str, getNewFeaturesString(), index, PLearn::TVec< T >::length(), PLearn::TVec< T >::push_back(), PLearn::TVec< T >::resize(), and string_to_index_feats.
{
int index;
string str;
getNewFeaturesString(token,f_str);
feats.resize(0);
for(int t=0; t<f_str.length(); t++)
{
str = f_str[t];
if(string_to_index_feats.find(str) != string_to_index_feats.end())
{
index = string_to_index_feats[str];
feats.push_back(index);
}
}
}

| int PLearn::HashMapFeatureSet::getIndexFeature | ( | string | str | ) | [virtual] |
Gives index form of a feature in string form.
Implements PLearn::FeatureSet.
Definition at line 129 of file HashMapFeatureSet.cc.
References PLERROR, and string_to_index_feats.
{
if(string_to_index_feats.find(str) == string_to_index_feats.end())
PLERROR("In HashMapFeatureSet::getIndexFeature(): string %s is an invalid feature", str.c_str());
return string_to_index_feats[str];
}
| virtual void PLearn::HashMapFeatureSet::getNewFeaturesString | ( | string | token, |
| TVec< string > & | feats_str | ||
| ) | [pure virtual] |
Gives the possibly new features in string form for a token.
Implements PLearn::FeatureSet.
Implemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Referenced by addFeatures(), and getFeatures().

| string PLearn::HashMapFeatureSet::getStringFeature | ( | int | index | ) | [virtual] |
Gives string form of a feature in index form.
Implements PLearn::FeatureSet.
Definition at line 119 of file HashMapFeatureSet.cc.
References index, index_to_string_feats, and PLERROR.
{
#ifdef BOUNDCHECK
if(index_to_string_feats.find(index) == index_to_string_feats.end())
PLERROR("In HashMapFeatureSet::getStringFeature(): index %d is an invalid feature index", index);
#endif
return index_to_string_feats[index];
}
| void PLearn::HashMapFeatureSet::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 72 of file HashMapFeatureSet.cc.
References PLearn::deepCopyField(), index_to_string_feats, PLearn::FeatureSet::makeDeepCopyFromShallowCopy(), and string_to_index_feats.
Referenced by PLearn::WordNetFeatureSet::makeDeepCopyFromShallowCopy(), and PLearn::PythonFeatureSet::makeDeepCopyFromShallowCopy().
{
inherited::makeDeepCopyFromShallowCopy(copies);
deepCopyField(index_to_string_feats, copies);
deepCopyField(string_to_index_feats, copies);
//PLERROR("HashMapFeatureSet::makeDeepCopyFromShallowCopy not fully (correctly) implemented yet!");
}


| int PLearn::HashMapFeatureSet::size | ( | ) | [virtual] |
Gives the number of features in the set.
Implements PLearn::FeatureSet.
Definition at line 137 of file HashMapFeatureSet.cc.
References index_to_string_feats.
{
return int(index_to_string_feats.size());
}
Reimplemented from PLearn::FeatureSet.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 81 of file HashMapFeatureSet.h.
TVec<string> PLearn::HashMapFeatureSet::f_str [private] |
Temporary computations vectors.
Reimplemented in PLearn::IdentityFeatureSet, PLearn::PythonFeatureSet, and PLearn::WordNetFeatureSet.
Definition at line 138 of file HashMapFeatureSet.h.
Referenced by addFeatures(), and getFeatures().
int PLearn::HashMapFeatureSet::index [private] |
Definition at line 139 of file HashMapFeatureSet.h.
Referenced by addFeatures(), getFeatures(), and getStringFeature().
hash_map<int,string> PLearn::HashMapFeatureSet::index_to_string_feats [protected] |
Index feature to string feature mapping.
Definition at line 118 of file HashMapFeatureSet.h.
Referenced by addFeatures(), clear(), declareOptions(), getStringFeature(), makeDeepCopyFromShallowCopy(), and size().
Indication that progress should be reported in a ProgressBar when filling the feature set.
Definition at line 70 of file HashMapFeatureSet.h.
Referenced by addFeatures(), and declareOptions().
hash_map<string,int> PLearn::HashMapFeatureSet::string_to_index_feats [protected] |
String feature to index feature mapping.
Definition at line 120 of file HashMapFeatureSet.h.
Referenced by addFeatures(), clear(), declareOptions(), getFeatures(), getIndexFeature(), and makeDeepCopyFromShallowCopy().
1.7.4