PLearn 0.1
Public Member Functions | Public Attributes | Protected Member Functions
PLearn::Cache< KeyType, ValueType > Class Template Reference

Class description: More...

#include <Cache.h>

Inheritance diagram for PLearn::Cache< KeyType, ValueType >:
Inheritance graph
[legend]
Collaboration diagram for PLearn::Cache< KeyType, ValueType >:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 Cache (string dir="cache", int max_memory_=0, bool singlefile=false)
const ValueType * operator() (const KeyType &key) const
 Try to get value associataed with key.
ValueType * operator() (const KeyType &key)
bool isCached (const KeyType &key) const
 Check if this key is in cache. This does not change the access priority of the key.
bool isInMemory (const KeyType &key) const
string filename (const KeyType &key) const
bool isOnFile (const KeyType &key) const
void clear ()
virtual ~Cache ()
virtual void synchronizeDisk () const

Public Attributes

bool single_file
string files_directory
PStream::mode_t file_format

Protected Member Functions

ValueType * loadValue (const KeyType &key)
void saveValue (const KeyType &key, const ValueType &value) const
void removeExcess ()
 remove last element until current_memory <= max_memory;
void removeAll ()
 remove all elements

Detailed Description

template<class KeyType, class ValueType>
class PLearn::Cache< KeyType, ValueType >

Class description:

Similar semantically to a map<KeyType,ValueType>, except that a maximum memory usage can be set, and less frequently and less recently accessed elements that make memory usage go above that limit are saved to a file (either a single large file, only if elements all have the size size, or one file per element). The memory cache is implemented with a BoundedMemoryCache, to which this class adds the ability to store the less frequently accessed items on disk. The elements keys and values (of template types KeyType and ValueType respectively) must be serializable with PLearn's PStream's. If one file per entry is used, the KeyType must be convertible to a string with 'tostring(key)', which will be used as a suffix of the file name (before the .psave suffix). ONLY THE FILES ASSOCIATED TO THIS CACHE SHOULD BE STORED IN THE files_directory.

Definition at line 75 of file Cache.h.


Constructor & Destructor Documentation

template<class KeyType , class ValueType >
PLearn::Cache< KeyType, ValueType >::Cache ( string  dir = "cache",
int  max_memory_ = 0,
bool  singlefile = false 
) [inline]

Definition at line 83 of file Cache.h.

        : BoundedMemoryCache<KeyType,ValueType>(max_memory_), single_file(singlefile), files_directory(dir), file_format(PStream::plearn_ascii)
    {}
template<class KeyType , class ValueType >
virtual PLearn::Cache< KeyType, ValueType >::~Cache ( ) [inline, virtual]

Definition at line 128 of file Cache.h.

References PLearn::clear().

Here is the call graph for this function:


Member Function Documentation

template<class KeyType , class ValueType >
void PLearn::Cache< KeyType, ValueType >::clear ( ) [inline]

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 117 of file Cache.h.

References PLearn::clear(), i, PLearn::lsdir(), and PLearn::rm().

                        {
        BoundedMemoryCache<KeyType,ValueType>::clear();
        if (files_directory!="")
        {
            vector<string> filenames = lsdir(files_directory);
            for (unsigned int i=0;i<filenames.size();i++)
                rm(files_directory + "/" + filenames[i]);
        }
    }

Here is the call graph for this function:

template<class KeyType , class ValueType >
string PLearn::Cache< KeyType, ValueType >::filename ( const KeyType &  key) const [inline]

Definition at line 114 of file Cache.h.

References PLearn::tostring().

{ return files_directory + "/" + tostring(key) + ".psave"; }

Here is the call graph for this function:

template<class KeyType , class ValueType >
bool PLearn::Cache< KeyType, ValueType >::isCached ( const KeyType &  key) const [inline]

Check if this key is in cache. This does not change the access priority of the key.

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 112 of file Cache.h.

template<class KeyType , class ValueType >
bool PLearn::Cache< KeyType, ValueType >::isInMemory ( const KeyType &  key) const [inline]

Definition at line 113 of file Cache.h.

template<class KeyType , class ValueType >
bool PLearn::Cache< KeyType, ValueType >::isOnFile ( const KeyType &  key) const [inline]

Definition at line 115 of file Cache.h.

References PLearn::isfile().

{ return isfile(filename(key)); }

Here is the call graph for this function:

template<class KeyType , class ValueType >
ValueType* PLearn::Cache< KeyType, ValueType >::loadValue ( const KeyType &  key) [inline, protected]

Definition at line 141 of file Cache.h.

References PLearn::isfile(), PLearn::openFile(), and PLERROR.

                                                    {
        if (single_file)
        {
            PLERROR("Cache: single_file mode not yet implemented");
        }
        else
        {
            string fname = filename(key);
            if (!isfile(fname))
                return 0;
            PStream filestream = openFile(fname, file_format, "r");
            ValueType* v = new ValueType;
            filestream >> *v;
            return v;
        }
        return 0;
    }

Here is the call graph for this function:

template<class KeyType , class ValueType >
ValueType* PLearn::Cache< KeyType, ValueType >::operator() ( const KeyType &  key) [inline]

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 100 of file Cache.h.

                                                     { 
        ValueType* value_p = BoundedMemoryCache<KeyType,ValueType>::operator()(key);
        if (value_p) // we have it in memory
            return value_p;
        // else check if we have it on file
        value_p = loadValue(key);
        if (value_p)
            BoundedMemoryCache<KeyType,ValueType>::set(key,*value_p);
        return value_p;
    }
template<class KeyType , class ValueType >
const ValueType* PLearn::Cache< KeyType, ValueType >::operator() ( const KeyType &  key) const [inline]

Try to get value associataed with key.

If not in cache (either memory or disk) return 0, else return pointer to value. Recently accessed keys (with set or operator()) are less likely to be removed from memory (saved to disk).

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 90 of file Cache.h.

                                                                 { 
        ValueType* value_p = BoundedMemoryCache<KeyType,ValueType>::operator()(key);
        if (value_p) // we have it in memory
            return value_p;
        // else check if we have it on file
        value_p = loadValue(key);
        if (value_p)
            BoundedMemoryCache<KeyType,ValueType>::set(key,*value_p);
        return value_p;
    }
template<class KeyType , class ValueType >
void PLearn::Cache< KeyType, ValueType >::removeAll ( ) [inline, protected, virtual]

remove all elements

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 192 of file Cache.h.

    {
        while (BoundedMemoryCache<KeyType,ValueType>::n_elements)
        {
            KeyType& key = BoundedMemoryCache<KeyType,ValueType>::doubly_linked_list->last->entry;

            // STORE THE ELEMENT TO BE DELETED ON DISK:
            pair<ValueType,DoublyLinkedListElement<KeyType>*> v = BoundedMemoryCache<KeyType,ValueType>::elements[key];
            saveValue(key, v.first);

//            BoundedMemoryCache<KeyType,ValueType>::current_memory -= sizeInBytes(BoundedMemoryCache<KeyType,ValueType>::elements[key]);
            v.second=0;  // this is just to help debugging, really, to help track wrong pointers
            BoundedMemoryCache<KeyType,ValueType>::elements.erase(key);
            BoundedMemoryCache<KeyType,ValueType>::doubly_linked_list->removeLast();
            BoundedMemoryCache<KeyType,ValueType>::n_elements--;
        }

        BoundedMemoryCache<KeyType,ValueType>::current_memory = 0;
        //int left_elements = BoundedMemoryCache<KeyType,ValueType>::n_elements;
    }
template<class KeyType , class ValueType >
void PLearn::Cache< KeyType, ValueType >::removeExcess ( ) [inline, protected, virtual]

remove last element until current_memory <= max_memory;

Reimplemented from PLearn::BoundedMemoryCache< KeyType, ValueType >.

Definition at line 171 of file Cache.h.

References PLearn::sizeInBytes().

    {
        while (BoundedMemoryCache<KeyType,ValueType>::current_memory > BoundedMemoryCache<KeyType,ValueType>::max_memory)
        {
            KeyType& key = BoundedMemoryCache<KeyType,ValueType>::doubly_linked_list->last->entry;

            // STORE THE ELEMENT TO BE DELETED ON DISK:
            pair<ValueType,DoublyLinkedListElement<KeyType>*> v = BoundedMemoryCache<KeyType,ValueType>::elements[key];
            saveValue(key, v.first);

            BoundedMemoryCache<KeyType,ValueType>::current_memory -= sizeInBytes(BoundedMemoryCache<KeyType,ValueType>::elements[key]);
            v.second=0;  // this is just to help debugging, really, to help track wrong pointers
            BoundedMemoryCache<KeyType,ValueType>::elements.erase(key);
            BoundedMemoryCache<KeyType,ValueType>::doubly_linked_list->removeLast();
            BoundedMemoryCache<KeyType,ValueType>::n_elements--;
        }

        //int left_elements = BoundedMemoryCache<KeyType,ValueType>::n_elements;
    }

Here is the call graph for this function:

template<class KeyType , class ValueType >
void PLearn::Cache< KeyType, ValueType >::saveValue ( const KeyType &  key,
const ValueType &  value 
) const [inline, protected]

Definition at line 158 of file Cache.h.

References PLearn::openFile(), and PLERROR.

                                                                            {
        if (single_file)
        {
            PLERROR("Cache: single_file mode not yet implemented");
        }
        else
        {
            PStream filestream = openFile(filename(key), file_format, "w");
            filestream << value;
        }
    }

Here is the call graph for this function:

template<class KeyType , class ValueType >
virtual void PLearn::Cache< KeyType, ValueType >::synchronizeDisk ( ) const [inline, virtual]

Definition at line 133 of file Cache.h.

                                                {
        typename map<KeyType,pair<ValueType,DoublyLinkedListElement<KeyType>*> >::const_iterator it = BoundedMemoryCache<KeyType,ValueType>::elements.begin();
        typename map<KeyType,pair<ValueType,DoublyLinkedListElement<KeyType>*> >::const_iterator end = BoundedMemoryCache<KeyType,ValueType>::elements.end();
        for (;it!=end;++it)
            saveValue(it->first,it->second.first);
    }

Member Data Documentation

template<class KeyType , class ValueType >
PStream::mode_t PLearn::Cache< KeyType, ValueType >::file_format

Definition at line 81 of file Cache.h.

template<class KeyType , class ValueType >
string PLearn::Cache< KeyType, ValueType >::files_directory

Definition at line 80 of file Cache.h.

template<class KeyType , class ValueType >
bool PLearn::Cache< KeyType, ValueType >::single_file

Definition at line 79 of file Cache.h.


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines