PLearn 0.1
Public Member Functions | Protected Member Functions | Protected Attributes | Static Protected Attributes
PLearn::IntVecFile Class Reference

#include <IntVecFile.h>

Inheritance diagram for PLearn::IntVecFile:
Inheritance graph
[legend]
Collaboration diagram for PLearn::IntVecFile:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 IntVecFile ()
 Default constructor, you must then call open.
 IntVecFile (const string &the_filename, bool readwrite=false)
 IntVecFile (const IntVecFile &other)
 The copy constructor opens the file a second time in readonly mode only.
virtual void open (const string &the_filename, bool readwrite=false)
virtual void close ()
virtual int get (int i) const
virtual void put (int i, int value)
TVec< intgetVec () const
const string & getFilename () const
int length () const
virtual int operator[] (int i) const
virtual void append (int value)
virtual void append (const TVec< int > &vec)
virtual ~IntVecFile ()

Protected Member Functions

void writeFileSignature ()
 write magic signature
void getVersionAndSize ()
 store in data members
void seek_to_index (int index) const
 seek depending on version

Protected Attributes

string filename
FILE * f
int length_
int version_number_
 0 if old version, 1 if
char endianness_
 either 'L' or 'B'

Static Protected Attributes

static const char signature []
 magic signature
static const int header_size []
 index array by version number

Detailed Description

IntVecFile is a class to handle a simple linear file of integers with random access.

There are two versions of the on-disk file format. The "old" version 0, where integers are stored in the file in plain little-endian binary representation, with no header at all. The "length" of the vector is inferred directly from the size of the file.

The version 1 has the following layout. The file starts with the magic header {0xDE, 0xAD, 0xBE, 0xEF}. Following is a single character: 'L' for little-endian, and 'B' for big-endian file. Finally, three bytes {0x00, 0x00, 0x01} (version 1) of the file format. As with version 0, the length of the vector is inferred from the size of the file.

Random access in both read and write are possible. And it is possible to write beyond length(), in which case length() will be updated to reflect the change

Definition at line 77 of file IntVecFile.h.


Constructor & Destructor Documentation

PLearn::IntVecFile::IntVecFile ( ) [inline]

Default constructor, you must then call open.

Definition at line 92 of file IntVecFile.h.

PLearn::IntVecFile::IntVecFile ( const string &  the_filename,
bool  readwrite = false 
) [inline]

Definition at line 96 of file IntVecFile.h.

        : f(0)  { open(the_filename, readwrite); }
PLearn::IntVecFile::IntVecFile ( const IntVecFile other)

The copy constructor opens the file a second time in readonly mode only.

Definition at line 62 of file IntVecFile.cc.

References filename, and open().

    : filename(other.filename), f(0), length_(other.length_),
      version_number_(other.version_number_), endianness_(other.endianness_)
{
    open(filename, false /* readonly */);
}

Here is the call graph for this function:

PLearn::IntVecFile::~IntVecFile ( ) [virtual]

Definition at line 119 of file IntVecFile.cc.

References close().

{
    close();
}

Here is the call graph for this function:


Member Function Documentation

virtual void PLearn::IntVecFile::append ( int  value) [inline, virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 113 of file IntVecFile.h.

Referenced by PLearn::filter(), PLearn::VVMatrix::generateFilterIndexFile(), PLearn::VVMatrix::generateVMatIndex(), PLearn::grep(), and PLearn::FilteredVMatrix::openIndex().

{ put(length(),value); }

Here is the caller graph for this function:

void PLearn::IntVecFile::append ( const TVec< int > &  vec) [virtual]

Definition at line 144 of file IntVecFile.cc.

References PLearn::byte_order(), PLearn::TVec< T >::data(), endianness_, PLearn::endianswap(), f, PLearn::TVec< T >::length(), length(), length_, and seek_to_index().

{
    seek_to_index(length());

    if (byte_order() != endianness_) {
        TVec<int> new_vec(vec.length());
        new_vec << vec;
        endianswap(new_vec.data(), new_vec.length());
        fwrite(new_vec.data(), sizeof(int), new_vec.length(), f);
    }
    else {
        fwrite(vec.data(), sizeof(int), vec.length(), f);
    }

    length_ += vec.length();
}

Here is the call graph for this function:

void PLearn::IntVecFile::close ( ) [virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 112 of file IntVecFile.cc.

References f.

Referenced by PLearn::VVMatrix::generateFilterIndexFile(), open(), PLearn::FilteredVMatrix::openIndex(), and ~IntVecFile().

{
    if(f)
        fclose(f);
    f=0;
}

Here is the caller graph for this function:

int PLearn::IntVecFile::get ( int  i) const [virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 94 of file IntVecFile.cc.

References BIG_ENDIAN_ORDER, endianness_, f, PLearn::fread_int(), length(), PLERROR, and seek_to_index().

{
#ifdef BOUNDCHECK
    if(i<0 || i>=length())
        PLERROR("Out Of Bounds in IntVecFile::get");
#endif
    seek_to_index(i);
    return fread_int(f, endianness_ == BIG_ENDIAN_ORDER);
}

Here is the call graph for this function:

const string& PLearn::IntVecFile::getFilename ( ) const [inline]

Definition at line 110 of file IntVecFile.h.

{ return filename; }
TVec< int > PLearn::IntVecFile::getVec ( ) const

Definition at line 127 of file IntVecFile.cc.

References PLearn::byte_order(), PLearn::TVec< T >::data(), endianness_, PLearn::endianswap(), f, length(), PLERROR, and seek_to_index().

Referenced by PLearn::VVMatrix::generateVMatIndex().

{
    size_t tt;
    TVec<int> res(length());
    seek_to_index(0);
    if((tt=fread(res.data(), sizeof(int), length(), f)) != size_t(length()))
        PLERROR("fread error in IntVecFile::getVec()");
    // Switch byte order if necessary
    if (byte_order() != endianness_)
        endianswap(res.data(), length());

    return res;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::IntVecFile::getVersionAndSize ( ) [protected]

store in data members

< unbelievable (bis)

Definition at line 177 of file IntVecFile.cc.

References BIG_ENDIAN_ORDER, endianness_, f, filename, PLearn::filesize(), header_size, i, length_, LITTLE_ENDIAN_ORDER, PLERROR, signature, and version_number_.

Referenced by open().

{
    if (sizeof(int) != 4)
        PLERROR("IntVecFile::getVersionAndSize: "
                "IntVecFile not yet designed to handle sizeof(int) != 4");
  
    long the_filesize = filesize(filename);
    if (the_filesize < long(2*sizeof(int)) /* assume 4 */)
        goto version0;                           // unbelievable but true!

    fseek(f, 0, SEEK_SET);
    for (int i=0; i<4; ++i)
        if (char(fgetc(f)) != signature[i])
            goto version0;                         

    // Assume new-world file format
    endianness_ = char(fgetc(f));
    if (endianness_ != LITTLE_ENDIAN_ORDER &&
        endianness_ != BIG_ENDIAN_ORDER)
        PLERROR("IntVecFile::getVersionAndSize: "
                "File format error in file %s Only supported endianness is 'L' or 'B'\n"
                "Read %c", filename.c_str(),endianness_);

    if(fgetc(f) != 0x00 || fgetc(f) != 0x00)
        PLERROR("IntVecFile::getVersionAndSize: "
                "File format error in file %s", filename.c_str());

    version_number_ = (unsigned char)fgetc(f);

    if (version_number_ > 1)
        PLERROR("IntVecFile::getVersionAndSize: "
                "File version (%d) is not supported", version_number_);

    length_ = int(the_filesize / sizeof(int) - header_size[version_number_]);
    return;

 version0:
    length_ = int(the_filesize / sizeof(int));
    version_number_ = 0;
    endianness_ = 'L';
}

Here is the call graph for this function:

Here is the caller graph for this function:

int PLearn::IntVecFile::length ( ) const [inline]
void PLearn::IntVecFile::open ( const string &  the_filename,
bool  readwrite = false 
) [virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 69 of file IntVecFile.cc.

References close(), f, filename, getVersionAndSize(), PLearn::isfile(), PLERROR, and writeFileSignature().

Referenced by PLearn::SelectRowsFileIndexVMatrix::build_(), IntVecFile(), and PLearn::FilteredVMatrix::openIndex().

{
    if(f)
        close();

    filename = the_filename;
    bool file_exists = isfile(filename);
    if(readwrite)
    {
        f = fopen(filename.c_str(),"a+b");
        if(!f)
            PLERROR("Couldn't open file %s for read/write",filename.c_str());
    }
    else
    {
        f = fopen(filename.c_str(),"rb");
        if(!f)
            PLERROR("Couldn't open file %s for reading",filename.c_str());
    }
    if (file_exists)
        getVersionAndSize();
    else
        writeFileSignature();
}

Here is the call graph for this function:

Here is the caller graph for this function:

virtual int PLearn::IntVecFile::operator[] ( int  i) const [inline, virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 112 of file IntVecFile.h.

References i.

{ return get(i); }
void PLearn::IntVecFile::put ( int  i,
int  value 
) [virtual]

Reimplemented in PLearn::BufferedIntVecFile.

Definition at line 104 of file IntVecFile.cc.

References BIG_ENDIAN_ORDER, endianness_, f, PLearn::fwrite_int(), length_, and seek_to_index().

Referenced by PLearn::fullyRebalance2Classes(), and PLearn::rebalanceNClasses().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::IntVecFile::seek_to_index ( int  index) const [protected]

seek depending on version

Definition at line 219 of file IntVecFile.cc.

References f, header_size, and version_number_.

Referenced by append(), get(), getVec(), and put().

{
    fseek(f, (i+header_size[version_number_]) * sizeof(int), SEEK_SET);
}

Here is the caller graph for this function:

void PLearn::IntVecFile::writeFileSignature ( ) [protected]

write magic signature

< write without \0

Definition at line 161 of file IntVecFile.cc.

References PLearn::byte_order(), endianness_, f, length_, signature, and version_number_.

Referenced by open().

{
    // This is for a new file.  Assume length 0, version number 1,
    // file endianness is current-platform endianness
    length_ = 0;
    version_number_ = 1;
    endianness_ = byte_order();

    fseek(f, 0, SEEK_SET);
    fputs(signature, f);                       
    fputc(endianness_, f);
    fputc(0x00, f);
    fputc(0x00, f);
    fputc(char(version_number_), f);
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

either 'L' or 'B'

Definition at line 85 of file IntVecFile.h.

Referenced by append(), get(), getVec(), getVersionAndSize(), put(), and writeFileSignature().

FILE* PLearn::IntVecFile::f [protected]
string PLearn::IntVecFile::filename [protected]

Definition at line 80 of file IntVecFile.h.

Referenced by getVersionAndSize(), IntVecFile(), and open().

const int PLearn::IntVecFile::header_size [static, protected]
Initial value:
 {      
    0,                                         
    2                                          
}

index array by version number

Definition at line 88 of file IntVecFile.h.

Referenced by getVersionAndSize(), and seek_to_index().

Definition at line 82 of file IntVecFile.h.

Referenced by append(), getVersionAndSize(), put(), and writeFileSignature().

const char PLearn::IntVecFile::signature [static, protected]
Initial value:
 {
    '\xDE', '\xAD', '\xBE', '\xEF', '\x00'     
}

magic signature

Definition at line 87 of file IntVecFile.h.

Referenced by getVersionAndSize(), and writeFileSignature().

0 if old version, 1 if

current version

Definition at line 83 of file IntVecFile.h.

Referenced by getVersionAndSize(), seek_to_index(), and writeFileSignature().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines