PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Static Protected Attributes | Private Types
PLearn::CompactVMatrix Class Reference

#include <CompactVMatrix.h>

Inheritance diagram for PLearn::CompactVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::CompactVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

int nbits ()
int nsymbols ()
int nfixedpoint ()
void setOneHotMode (bool on=true)
 CompactVMatrix ()
 default constructor (for automatic deserialization)
 CompactVMatrix (int the_length, int n_variables, int n_binary, int n_nonbinary_discrete, int n_fixed_point, TVec< int > &n_symbolvalues, Vec &fixed_point_min, Vec &fixed_point_max, bool one_hot_encoding=true)
 CompactVMatrix (VMat m, int keep_last_variables_last=1, bool onehot_encoding=true)
 CompactVMatrix (const string &filename, int nlast=1)
 construct from saved CompactVMatrix
 CompactVMatrix (CompactVMatrix *cvm, VMat m, bool rescale=false, bool check=true)
void append (CompactVMatrix *vm)
 append vm to this VMatrix (the rows of vm are concatenated to the current rows of this VMatrix)
void perturb (int i, Vec row, real noise_level, int n_last)
TVec< int > & permutation_vector ()
virtual real squareDifference (int i, int j)
 encoding (v is not one-hot, and the variables in v are in the "original" order
virtual real dotProduct (int i, int j) const
 return the dot product of row i with row j, excluding n_last columns
virtual real dot (int i1, int i2, int inputsize) const
 Returns the dot product between row i1 and row i2 (considering only the inputsize first elements).
virtual real dot (int i, const Vec &v) const
 Returns the result of the dot product between row i and the given vec (only v.length() first elements of row i are considered).
virtual void encodeAndPutRow (int i, Vec v)
 (i.e. at position i in v we find variable variables_permutation[i] in getRow's result)
virtual void putRow (int i, Vec v)
 v is possibly one-hot-encoded (according to one_hot_encoding flag) and the variables are in the same order as for getRow.
virtual void putSubRow (int i, int j, Vec v)
 It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead)
virtual void save (const PPath &filename) const
 save everything in file (data and auxiliary information), binary format
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual CompactVMatrixdeepCopy (CopiesMap &copies) const
void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual void build ()
 nothing to do...

Static Public Member Functions

static string _classname_ ()
 reverse of write, can be used by calling load(string)
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

TVec< intn_symbol_values
 for each 1-byte symbol, the number of possible values
int n_last
 used by dotProduct and squareDifference to specify # of last columns to ignore

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 decoding (v may be one-hot depending on one_hot_encoding flag)

Static Protected Member Functions

static void set_n_bits_in_byte ()

Protected Attributes

Storage< unsigned char > data
 Each row of the matrix holds in order: bits, 1-byte symbols, fixed point numbers.
int row_n_bytes
 # of bytes per row
int n_bits
 number of binary symbols per row
int n_symbols
 number of 1-byte symbols per row
int n_fixedpoint
 number of fixed point numbers per row
int n_variables
 = n_bits + n_symbols + n_fixedpoint
bool one_hot_encoding
 the 1-byte symbols are converted to one-hot encoding by get
Vec fixedpoint_min
Vec fixedpoint_max
 the ranges of each number for fixed point encoding
Vec delta
 (fixedpoint_max-fixedpoint_min)/2^16
TVec< intvariables_permutation
 this variable is used only when constructed from VMat
int normal_width
 the value of width_ when one_hot_encoding=true
int symbols_offset
 auxiliary
int fixedpoint_offset
 where in each row the fixed point numbers start
Vec row_norms
 to cache the norms of the rows for squareDifference method

Static Protected Attributes

static unsigned char n_bits_in_byte [256]
 CompactVMatrix.

Private Types

typedef RowBufferedVMatrix inherited

Detailed Description

Like MemoryVMatrix this class holds the data in memory, but it tries to hold it compactly by using single bits for binary variables, single bytes for discrete variables whose number of possible values is less than 256, and unsigned shorts for the others, using a fixed point representation.

Definition at line 63 of file CompactVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 65 of file CompactVMatrix.h.


Constructor & Destructor Documentation

PLearn::CompactVMatrix::CompactVMatrix ( )

default constructor (for automatic deserialization)

Definition at line 79 of file CompactVMatrix.cc.

PLearn::CompactVMatrix::CompactVMatrix ( int  the_length,
int  n_variables,
int  n_binary,
int  n_nonbinary_discrete,
int  n_fixed_point,
TVec< int > &  n_symbolvalues,
Vec fixed_point_min,
Vec fixed_point_max,
bool  one_hot_encoding = true 
)

Definition at line 85 of file CompactVMatrix.cc.

References data, delta, fixedpoint_max, fixedpoint_min, fixedpoint_offset, i, PLearn::VMatrix::length_, n_bits, n_symbol_values, n_symbols, n_variables, normal_width, one_hot_encoding, PLearn::Storage< T >::resize(), row_n_bytes, set_n_bits_in_byte(), setOneHotMode(), symbols_offset, and variables_permutation.

    : inherited(the_length,nvariables), n_bits(n_binary),
      n_symbols(n_nonbinary_discrete), n_fixedpoint(n_fixed_point),
      n_variables(nvariables), one_hot_encoding(onehot_encoding),
      n_symbol_values(n_symbolvalues),
      fixedpoint_min(fixed_point_min), fixedpoint_max(fixed_point_max),
      delta(n_fixed_point), variables_permutation(n_variables)
{
    normal_width=n_bits+n_fixed_point;
    for (int i=0;i<n_symbols;i++)
        normal_width += n_symbol_values[i];
    setOneHotMode(one_hot_encoding);
    for (int i=0;i<n_variables;i++) variables_permutation[i]=i;
    for (int i=0;i<n_symbols;i++)
        delta[i]=(fixedpoint_max[i]-fixedpoint_min[i])/USHRT_MAX;
    symbols_offset = (int)ceil(n_bits/8.0);
    fixedpoint_offset = symbols_offset + n_symbols;
    row_n_bytes =  fixedpoint_offset + int(sizeof(unsigned short))*n_fixed_point;
    data.resize(length_ * row_n_bytes);
    set_n_bits_in_byte();
}

Here is the call graph for this function:

PLearn::CompactVMatrix::CompactVMatrix ( VMat  m,
int  keep_last_variables_last = 1,
bool  onehot_encoding = true 
)

Convert a VMat into a CompactVMatrix: this will use the stats computed in the fieldstats of the VMatrix (they will be computed if not already) to figure out which variables are binary, discrete (and how many symbols), and the ranges of numeric variables. THE VMAT DISCRETE VARIABLES MUST NOT BE ALREADY ONE-HOT ENCODED. The variables will be permuted according to the permutation vector which can be retrieved from the variables_permutation_vector() method. By default the last column of the VMat will stay last, thus being coded as fixedpoint (so the permutation information may not be necessary if the last column represents a target and all the previous ones some inputs. keep_last_variables_last is the number of "last columns" to keep in place.

Definition at line 111 of file CompactVMatrix.cc.

References PLearn::VMFieldStat::counts, data, delta, encodeAndPutRow(), PLearn::endl(), PLearn::fast_exact_is_equal(), PLearn::VMatrix::fieldinfos, PLearn::VMatrix::fieldstats, fixedpoint_max, fixedpoint_min, fixedpoint_offset, i, PLearn::isMapKeysAreInt(), j, PLearn::VMatrix::length_, PLearn::VMFieldStat::max(), PLearn::VMFieldStat::min(), n_bits, n_fixedpoint, n_symbol_values, n_symbols, n_variables, normal_width, one_hot_encoding, PLERROR, PLearn::Storage< T >::resize(), PLearn::TVec< T >::resize(), row_n_bytes, set_n_bits_in_byte(), setOneHotMode(), symbols_offset, variables_permutation, PLearn::VMat::width(), and PLearn::VMatrix::width_.

    : inherited(m->length(),m->width()), one_hot_encoding(onehot_encoding),
      n_symbol_values(m->width()), variables_permutation(m->width())
{
    if (!m->hasStats())
    {
        cout << "CompactVMatrix(VMat, int): VMat did not have stats. Computing them." << endl;
        m->computeStats();
    }
    // determine which variables are binary discrete, multi-class discrete, or continuous
    n_bits = n_symbols = n_fixedpoint = 0;
    TVec<int> bits_position(m->width());
    TVec<int> symbols_position(m->width());
    TVec<int> fp_position(m->width());
    fixedpoint_min.resize(m->width());
    fixedpoint_max.resize(m->width());
    delta.resize(m->width());
    for (int i=0;i<m->width();i++)
    {
        VMFieldStat& stat = m->fieldstats[i];
        int n_values = (int)stat.counts.size(); // 0 means "continuous"
        bool counts_look_continuous = !isMapKeysAreInt(stat.counts);
        if (n_values == 0 || counts_look_continuous || i>=m->width()-keep_last_variables_last)
        {
            fixedpoint_min[n_fixedpoint]=stat.min();
            fixedpoint_max[n_fixedpoint]=stat.max();
            delta[n_fixedpoint]=(stat.max()-stat.min())/USHRT_MAX;
            fp_position[n_fixedpoint++]=i;
        }
        else
        {
            if (!fast_exact_is_equal(stat.min(), 0) ||
                !fast_exact_is_equal((stat.max()-stat.min()+1),
                                     stat.counts.size()))
                PLERROR("CompactVMatrix:: variable %d looks discrete but has zero-frequency intermediate values or min!=0",i);
            if (n_values==2)
                bits_position[n_bits++]=i;
            else if (n_values<=256)
            {
                symbols_position[n_symbols]=i;
                n_symbol_values[n_symbols++] = n_values;
            }
            else
            {
                fixedpoint_min[n_fixedpoint]=stat.min();
                fixedpoint_max[n_fixedpoint]=stat.max();
                delta[n_fixedpoint]=(stat.max()-stat.min())/USHRT_MAX;
                fp_position[n_fixedpoint++]=i;
            }
        }
    }
    fixedpoint_min.resize(n_fixedpoint);
    fixedpoint_max.resize(n_fixedpoint);
    delta.resize(n_fixedpoint);
    n_symbol_values.resize(n_symbols);
    n_variables = n_bits + n_symbols + n_fixedpoint;
    int j=0;
    for (int i=0;i<n_bits;i++,j++)
        variables_permutation[j]=bits_position[i];
    for (int i=0;i<n_symbols;i++,j++)
        variables_permutation[j]=symbols_position[i];
    for (int i=0;i<n_fixedpoint;i++,j++)
        variables_permutation[j]=fp_position[i];

    normal_width=n_bits+n_fixedpoint;
    for (int i=0;i<n_symbols;i++)
        normal_width += n_symbol_values[i];
    setOneHotMode(one_hot_encoding);
    symbols_offset = (int)ceil(n_bits/8.0);
    fixedpoint_offset = symbols_offset + n_symbols;
    row_n_bytes =  fixedpoint_offset + int(sizeof(unsigned short))*n_fixedpoint;
    data.resize(length_ * row_n_bytes);

    // copy the field infos and stats? not really useful with one-hot encoding
    // because of non-binary symbols being spread across many columns.
    if (!one_hot_encoding)
    {
        fieldinfos.resize(width_);
        fieldstats.resize(width_);
        for (int i=0;i<width_;i++)
        {
            fieldinfos[i]=m->getFieldInfos()[variables_permutation[i]];
            fieldstats[i]=m->fieldstats[variables_permutation[i]];
        }
    }
    else
    {
        fieldinfos.resize(0);
        fieldstats.resize(0);
    }

    // copy the data
    Vec mrow(m->width());
    for (int t=0;t<length_;t++)
    {
        m->getRow(t,mrow);
        encodeAndPutRow(t,mrow);
    }
    set_n_bits_in_byte();
}

Here is the call graph for this function:

PLearn::CompactVMatrix::CompactVMatrix ( const string &  filename,
int  nlast = 1 
)

construct from saved CompactVMatrix

Definition at line 213 of file CompactVMatrix.cc.

References PLearn::Object::load(), n_last, and set_n_bits_in_byte().

                                                                : RowBufferedVMatrix(0,0)
{
    load(filename);
    n_last=nlast;
    set_n_bits_in_byte();
}

Here is the call graph for this function:

PLearn::CompactVMatrix::CompactVMatrix ( CompactVMatrix cvm,
VMat  m,
bool  rescale = false,
bool  check = true 
)

Create a CompactVMatrix with the same structure as cvcm but containing the data in m. Both must obviously have the same width. If rescale is true, then the min/max values for fixed-point encoding are recomputed. If check==true than this is verified and an error message is thrown if the floating point data are not in the expected ranges (of cvm).

Definition at line 220 of file CompactVMatrix.cc.

References PLearn::TVec< T >::copy(), data, delta, fixedpoint_max, fixedpoint_min, fixedpoint_offset, i, j, PLearn::VMatrix::length_, n_bits, n_fixedpoint, n_last, n_symbol_values, n_symbols, n_variables, normal_width, one_hot_encoding, PLERROR, putRow(), PLearn::Storage< T >::resize(), row_n_bytes, setOneHotMode(), symbols_offset, variables_permutation, PLearn::VMat::width(), PLearn::VMatrix::width(), and PLearn::VMatrix::width_.

    : inherited(m->length(),m->width())
{
    if (cvm->width() != m->width())
        PLERROR("CompactVMatrix::CompactVMatrix(CompactVMatrix* cvm, VMat m,...), args have widths %d!=%d",
                cvm->width(),m->width());
    // copy all the ordinary fields
    row_n_bytes = cvm->row_n_bytes;
    data.resize(length_*row_n_bytes);
    n_bits = cvm->n_bits;
    n_symbols = cvm->n_symbols;
    n_fixedpoint = cvm->n_fixedpoint;
    n_variables = cvm->n_variables;
    n_symbol_values = cvm->n_symbol_values;
    fixedpoint_min = cvm->fixedpoint_min.copy();
    fixedpoint_max = cvm->fixedpoint_max.copy();
    delta = cvm->delta.copy();
    variables_permutation = cvm->variables_permutation;
    n_last = cvm->n_last;
    normal_width = cvm->normal_width;
    symbols_offset = cvm->symbols_offset;
    fixedpoint_offset = cvm->fixedpoint_offset;

    setOneHotMode(cvm->one_hot_encoding);
    Vec row(width_);
    int offs=width_-n_fixedpoint;
    if (rescale)
    {
        for (int i=0;i<length_;i++)
        {
            m->getRow(i,row);
            for (int j=0;j<n_fixedpoint;j++)
            {
                real element=row[offs+j];
                if (element<fixedpoint_min[j])
                    fixedpoint_min[j]=element;
                if (element>fixedpoint_max[j])
                    fixedpoint_max[j]=element;
            }
        }
        for (int j=0;j<n_fixedpoint;j++)
            delta[j]=(fixedpoint_max[j]-fixedpoint_min[j])/USHRT_MAX;
    }
    for (int i=0;i<length_;i++)
    {
        m->getRow(i,row);
        if (!rescale && check) // check that range is OK
        {
            for (int j=0;j<n_fixedpoint;j++)
            {
                real element=row[offs+j];
                if (element<fixedpoint_min[j] ||
                    element>fixedpoint_max[j])
                    PLERROR("CompactVMatrix::CompactVMatrix(CompactVMatrix* cvm, VMat m,...) out-of-range element(%d,%d)=%g not in [%g,%g]",
                            i,j,element,fixedpoint_min[j],fixedpoint_max[j]);
            }
        }
        putRow(i,row);
    }
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::CompactVMatrix::_classname_ ( ) [static]

reverse of write, can be used by calling load(string)

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

OptionList & PLearn::CompactVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

RemoteMethodMap & PLearn::CompactVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

bool PLearn::CompactVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

Object * PLearn::CompactVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 77 of file CompactVMatrix.cc.

StaticInitializer CompactVMatrix::_static_initializer_ & PLearn::CompactVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

void PLearn::CompactVMatrix::append ( CompactVMatrix vm)

append vm to this VMatrix (the rows of vm are concatenated to the current rows of this VMatrix)

Definition at line 741 of file CompactVMatrix.cc.

References PLearn::TVec< T >::copy(), PLearn::Storage< T >::data, data, delta, PLearn::endl(), fixedpoint_max, fixedpoint_min, fixedpoint_offset, PLearn::RowBufferedVMatrix::getRow(), j, PLearn::VMatrix::length(), PLearn::VMatrix::length_, n_bits, n_fixedpoint, n_symbol_values, n_symbols, one_hot_encoding, PLERROR, putRow(), PLearn::Storage< T >::resize(), row_n_bytes, setOneHotMode(), PLearn::short_and_twobytes::twobytes, u, PLearn::short_and_twobytes::us, PLearn::VMatrix::width(), PLearn::VMatrix::width_, and PLearn::Object::write().

{
    if (width_!=vm->width())
        PLERROR("CompactVMatrix::append, incompatible width %d vs %d",
                width_,vm->width());
    if (row_n_bytes!=vm->row_n_bytes)
        PLERROR("CompactVMatrix::append, incompatible row_n_bytes %d vs %d",
                row_n_bytes,vm->row_n_bytes);
    if (n_bits!=vm->n_bits)
        PLERROR("CompactVMatrix::append, incompatible n_bits %d vs %d",
                n_bits,vm->n_bits);
    if (n_symbols!=vm->n_symbols)
        PLERROR("CompactVMatrix::append, incompatible n_symbols %d vs %d",
                n_symbols,vm->n_symbols);
    if (n_fixedpoint!=vm->n_fixedpoint)
        PLERROR("CompactVMatrix::append, incompatible n_fixedpoint %d vs %d",
                n_fixedpoint,vm->n_fixedpoint);
    if (n_symbol_values!=vm->n_symbol_values)
    {
        //n_symbol_values.write(cerr); cerr << endl;
        //vm->n_symbol_values.write(cerr); cerr << endl;
        PLearn::write(cerr, n_symbol_values);
        cerr << endl;
        PLearn::write(cerr, vm->n_symbol_values);
        cerr << endl;
        PLERROR("CompactVMatrix::append, incompatible n_symbol_values");
    }
    bool rescale = false;
    for (int j=0;j<n_fixedpoint && !rescale;j++)
        if (fixedpoint_min[j]>vm->fixedpoint_min[j] ||
            fixedpoint_max[j]<vm->fixedpoint_max[j]) rescale=true;
    if (rescale)
    {
        cout << "The appended VMat has intervals that are wider than the current one." << endl;
        cout << "Start rescaling numeric variables fixed point representation." << endl;
        Vec new_min = fixedpoint_min.copy();
        Vec new_max = fixedpoint_max.copy();
        Vec new_delta = delta.copy();
        TVec<bool> change(n_fixedpoint);
        for (int j=0;j<n_fixedpoint;j++)
        {
            change[j]=false;
            if (fixedpoint_min[j]>vm->fixedpoint_min[j])
            {
                change[j]=true;
                new_min[j]=vm->fixedpoint_min[j];
            }
            if (fixedpoint_max[j]<vm->fixedpoint_max[j])
            {
                change[j]=true;
                new_max[j]=vm->fixedpoint_max[j];
            }
            if (change[j])
                new_delta[j]=(new_max[j]-new_min[j])/USHRT_MAX;
        }
        for (int r=0;r<length_;r++)
        {
            unsigned char* encoded_row = &data.data[r*row_n_bytes];
            unsigned char* fixed_point_numbers = &encoded_row[fixedpoint_offset];
            for (int j=0;j<n_fixedpoint;j++)
                if (change[j])
                {
                    // DECODE using previous min/max
                    unsigned char *uc = &fixed_point_numbers[2*j];
                    short_and_twobytes u;
                    u.twobytes[0]=uc[0];
                    u.twobytes[1]=uc[1];
                    real decoded = u.us*delta[j]+fixedpoint_min[j];
                    // correct rounding errors for integers, due to fixed-point low precision
                    real rounded_decoded = rint(decoded);
                    if (fabs(rounded_decoded-decoded)<1e-4)
                        decoded = rounded_decoded;
                    // ENCODE using new min/max
                    fixed_point_numbers[j]=(unsigned char)((decoded-new_min[j])/new_delta[j]);
                }
        }
        cout << "DONE rescaling numeric variables fixed point representation." << endl;
        fixedpoint_min << new_min;
        fixedpoint_max << new_max;
        delta << new_delta;
    }
    int new_length=length_+vm->length();
    data.resize(row_n_bytes*new_length);
    // copy the new data
    Vec row(width_);
    bool old_vm_encoding = vm->one_hot_encoding;
    bool old_encoding = one_hot_encoding;
    vm->one_hot_encoding=false;
    setOneHotMode(false);
    int old_length=length_;
    length_=new_length;
    for (int r=0;r<vm->length();r++)
    {
        vm->getRow(r,row);
        putRow(old_length+r,row);
    }
    vm->one_hot_encoding=old_vm_encoding;
    setOneHotMode(old_encoding);
}

Here is the call graph for this function:

virtual void PLearn::CompactVMatrix::build ( ) [inline, virtual]

nothing to do...

Reimplemented from PLearn::VMatrix.

Definition at line 194 of file CompactVMatrix.h.

string PLearn::CompactVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 77 of file CompactVMatrix.cc.

static const PPath& PLearn::CompactVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 192 of file CompactVMatrix.h.

{}  
CompactVMatrix * PLearn::CompactVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 77 of file CompactVMatrix.cc.

real PLearn::CompactVMatrix::dot ( int  i,
const Vec v 
) const [virtual]

Returns the result of the dot product between row i and the given vec (only v.length() first elements of row i are considered).

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 416 of file CompactVMatrix.cc.

References b, c, PLearn::TVec< T >::data(), PLearn::Storage< T >::data, data, delta, PLearn::diff(), dot(), PLearn::dot_product(), fixedpoint_min, fixedpoint_offset, PLearn::RowBufferedVMatrix::getRow(), j, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, n, n_bits, n_fixedpoint, n_last, n_symbol_values, n_symbols, one_hot_encoding, PLERROR, row_n_bytes, SANITYCHECK_CompactVMatrix_PRECISION, symbols_offset, PLearn::short_and_twobytes::twobytes, u, PLearn::short_and_twobytes::us, and PLearn::VMatrix::width().

{
#ifdef BOUNDCHECK
    if (i<0 || i>=length_)
        PLERROR("CompactVMatrix::dot, row %d out of bounds [0,%d]",i,length_-1);
#endif

    if(v.length()!=width()-n_last)
        PLERROR("In CompactVMatrix::dot, in current implementation v.length() must be equal to width()-n_last");

    real dot_product = 0.;

    unsigned char* encoded_row = &data.data[i*row_n_bytes];
    real* vp=v.data();
    int c=0;
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte=encoded_row[b];
        for (int j=0;j<8 && c<n_bits;j++,c++)
        {
            int bit = byte & 1;
            byte >>= 1; // shift right once
            if(bit)
                dot_product += vp[c];
        }
    }
    for (int b=0;b<n_symbols;b++)
    {
        int byte = encoded_row[symbols_offset+b];
        if (one_hot_encoding)
        {
            int n=n_symbol_values[b];
            dot_product += vp[c+byte];
            c += n;
        }
        else
            dot_product += vp[c++]*byte;
    }
    // WARNING: COULD THIS CAUSE PROBLEMS IF fixedpoint_offset IS NOT A MULTIPLE OF 4
    // ON SOME MACHINES?
    unsigned char* fixed_point_numbers = &encoded_row[fixedpoint_offset];
    for (int j=0;j<n_fixedpoint-n_last && c<v.length();j++,c++)
    {
        unsigned char *uc = &fixed_point_numbers[2*j];
        short_and_twobytes u;
        u.twobytes[0]=uc[0];
        u.twobytes[1]=uc[1];
        real decoded = u.us*delta[j]+fixedpoint_min[j];
        // correct rounding errors for integers, due to fixed-point low precision
        real rounded_decoded = rint(decoded);
        if (fabs(rounded_decoded-decoded)<1e-4)
            decoded = rounded_decoded;
        dot_product += vp[c] * decoded;
    }

    // Very Slow SANITY CHECK
#ifdef SANITYCHECK_CompactVMatrix
    Vec v_i(v.length());
    getRow(i,v_i);
    real dot_product2 = PLearn::dot(v_i.subVec(0,v.length()),v);
    real diff = fabs(dot_product-dot_product2)/fabs(dot_product2);
    if(diff>SANITYCHECK_CompactVMatrix_PRECISION)
        PLERROR("IN CompactVMatrix::dot(int i=%d, v) SANITY CHECK FAILED: difference=%g",i,diff);
#endif

    return dot_product;
}

Here is the call graph for this function:

real PLearn::CompactVMatrix::dot ( int  i1,
int  i2,
int  inputsize 
) const [virtual]

Returns the dot product between row i1 and row i2 (considering only the inputsize first elements).

The default version in VMatrix is somewhat inefficient, as it repeatedly calls get(i,j) The default version in RowBufferedVMatrix is a little better as it buffers the 2 Vecs between calls in case one of them is needed again. But the real strength of this method is for specialised and efficient versions in subbclasses. This method is typically used by SmartKernels so that they can compute kernel values between input samples efficiently.

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 342 of file CompactVMatrix.cc.

References b, c, PLearn::Storage< T >::data, data, delta, PLearn::dot_product(), fixedpoint_min, fixedpoint_offset, n_bits, n_bits_in_byte, n_fixedpoint, n_last, n_symbol_values, n_symbols, one_hot_encoding, PLERROR, row_n_bytes, symbols_offset, PLearn::short_and_twobytes::twobytes, u, PLearn::short_and_twobytes::us, PLearn::VMatrix::width(), and PLearn::VMatrix::width_.

Referenced by dot(), and dotProduct().

{
    if(inputsize!=width()-n_last)
        PLERROR("In CompactVMatrix::dot, in current implementation inputsize must be equal to width()-n_last");

    unsigned char* encoded_row_i = &data.data[i*row_n_bytes];
    unsigned char* encoded_row_j = &data.data[j*row_n_bytes];
    real dot_product=0.;
    int c=0;
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte_i=encoded_row_i[b];
        unsigned char byte_j=encoded_row_j[b];
        unsigned char byte_and = byte_i & byte_j;
#ifdef SANITYCHECK_CompactVMatrix
        real check=dot_product;
#endif
        // Here we want to count the number of ON bits in the byte_and
        // instead of looping through the bits, we look-up in a
        // pre-computed table (n_bits_in_byte), which has been set-up by set_n_bits_in_byte().
        dot_product += n_bits_in_byte[byte_and];
#ifdef SANITYCHECK_CompactVMatrix
        for (int j=0;j<8 && c<n_bits;j++,c++)
        {
            check += byte_and & 1;
            byte_and >>= 1; // shift right once
        }
        if (check!=dot_product)
            PLERROR("logic error in n_bits_in_byte");
#else
        c+=8;
        if (c>n_bits) c=n_bits;
#endif
    }
    if (c>width_-n_last)
        PLERROR("CompactVMatrix: n_last should be among discrete non-binary or continuous variables");
    for (int b=0;b<n_symbols && c<width_-n_last;b++)
    {
        int byte_i = encoded_row_i[symbols_offset+b];
        int byte_j = encoded_row_j[symbols_offset+b];
        if (byte_i==byte_j) dot_product++;
        if (one_hot_encoding)
            c+=n_symbol_values[b];
        else
            c++;
    }
    unsigned char* fixed_point_numbers_i = &encoded_row_i[fixedpoint_offset];
    unsigned char* fixed_point_numbers_j = &encoded_row_j[fixedpoint_offset];
    for (int k=0;k<n_fixedpoint-n_last && c<width_-n_last;k++,c++)
    {
        unsigned char *uc = &fixed_point_numbers_i[2*k];
        short_and_twobytes u;
        u.twobytes[0]=uc[0];
        u.twobytes[1]=uc[1];
        real decoded_i = u.us*delta[k]+fixedpoint_min[k];
        uc = &fixed_point_numbers_j[2*k];
        u.twobytes[0]=uc[0];
        u.twobytes[1]=uc[1];
        real decoded_j = u.us*delta[k]+fixedpoint_min[k];
#ifdef SANITYCHECK_CompactVMatrix
        real rounded_decoded_i = rint(decoded_i);
        if (fabs(rounded_decoded_i-decoded_i)<1e-4)
            decoded_i = rounded_decoded_i;
        real rounded_decoded_j = rint(decoded_j);
        if (fabs(rounded_decoded_j-decoded_j)<1e-4)
            decoded_j = rounded_decoded_j;
#endif
        dot_product += decoded_i * decoded_j;
    }

    return dot_product;
}

Here is the call graph for this function:

Here is the caller graph for this function:

real PLearn::CompactVMatrix::dotProduct ( int  i,
int  j 
) const [virtual]

return the dot product of row i with row j, excluding n_last columns

Definition at line 485 of file CompactVMatrix.cc.

References dot(), n_last, and PLearn::VMatrix::width().

Referenced by squareDifference().

{ return dot(i,j,width()-n_last); }

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::CompactVMatrix::encodeAndPutRow ( int  i,
Vec  v 
) [virtual]

(i.e. at position i in v we find variable variables_permutation[i] in getRow's result)

Definition at line 499 of file CompactVMatrix.cc.

References b, c, data, PLearn::Storage< T >::data, PLearn::TVec< T >::data(), delta, PLearn::fast_exact_is_equal(), fixedpoint_min, fixedpoint_offset, PLearn::RowBufferedVMatrix::invalidateBuffer(), j, n_bits, n_fixedpoint, n_symbol_values, n_symbols, PLERROR, row_n_bytes, symbols_offset, and variables_permutation.

Referenced by CompactVMatrix().

{
    unsigned char* encoded_row = &data.data[i*row_n_bytes];
    real* vp=v.data();
    int* perm=variables_permutation.data();
    int c=0;
    // 1 vector element ==> 1 bit
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte=0;
        for (int j=0;j<8 && c<n_bits;j++,c++)
            byte |= int(vp[perm[c]]) << j; // shift to right bit position
        encoded_row[b]=byte;
    }
    //    1 vector element (integer between 0 and n-1) ==> 1 byte
    for (int b=0;b<n_symbols;b++,c++)
    {
        real val = vp[perm[c]];
        int s = int(val);
        if (!fast_exact_is_equal(s, val))
            PLERROR("CompactVMatrix::encodeAndPutRow(%d,v): v[%d]=%g not an integer",
                    i,int(perm[c]),val);
        encoded_row[symbols_offset+b] = s; // ASSUMES THAT v IS NOT ONE-HOT ENCODED
        if (s<0 || s>=n_symbol_values[b])
            PLERROR("CompactVMatrix::encodeAndPutRow(%d,v): v[%d]=%d not in expected range (0,%d)",
                    i,int(perm[c]),s,n_symbol_values[b]-1);
    }
    // WARNING: COULD THIS CAUSE PROBLEMS IF fixedpoint_offset IS NOT A MULTIPLE OF 4
    // ON SOME MACHINES?
    unsigned short* fixed_point_numbers = (unsigned short*)&encoded_row[fixedpoint_offset];
    for (int j=0;j<n_fixedpoint;j++,c++)
        fixed_point_numbers[j]=(unsigned short)((vp[perm[c]]-fixedpoint_min[j])/delta[j]);

    invalidateBuffer();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::CompactVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

decoding (v may be one-hot depending on one_hot_encoding flag)

Implements PLearn::RowBufferedVMatrix.

Definition at line 290 of file CompactVMatrix.cc.

References b, c, data, PLearn::Storage< T >::data, PLearn::TVec< T >::data(), delta, fixedpoint_min, fixedpoint_offset, j, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, n, n_bits, n_fixedpoint, n_symbol_values, n_symbols, one_hot_encoding, PLERROR, row_n_bytes, symbols_offset, PLearn::short_and_twobytes::twobytes, u, PLearn::short_and_twobytes::us, PLearn::VMatrix::width(), and PLearn::VMatrix::width_.

{
#ifdef BOUNDCHECK
    if (i<0 || i>=length_)
        PLERROR("CompactVMatrix::getNewRow, row %d out of bounds [0,%d]",i,length_-1);
    if (v.length()!=width_)
        PLERROR("CompactVMatrix::getNewRow, length of v (%d) should be equal to width of VMat (%d)",v.length(),width());
#endif
    unsigned char* encoded_row = &data.data[i*row_n_bytes];
    real* vp=v.data();
    int c=0;
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte=encoded_row[b];
        for (int j=0;j<8 && c<n_bits;j++,c++)
        {
            int bit = byte & 1;
            byte >>= 1; // shift right once
            vp[c]=bit;
        }
    }
    for (int b=0;b<n_symbols;b++)
    {
        int byte = encoded_row[symbols_offset+b];
        if (one_hot_encoding)
        {
            int n=n_symbol_values[b];
            for (int j=0;j<n;j++) vp[c+j]=0;
            vp[c+byte]=1;
            c+=n;
        }
        else vp[c++]=byte;
    }
    unsigned char* fixed_point_numbers = &encoded_row[fixedpoint_offset];
    for (int j=0;j<n_fixedpoint;j++,c++)
    {
        unsigned char *uc = &fixed_point_numbers[2*j];
        short_and_twobytes u;
        u.twobytes[0]=uc[0];
        u.twobytes[1]=uc[1];
        real decoded = u.us*delta[j]+fixedpoint_min[j];
        // correct rounding errors for integers, due to fixed-point low precision
        real rounded_decoded = rint(decoded);
        if (fabs(rounded_decoded-decoded)<1e-4)
            decoded = rounded_decoded;
        vp[c]=decoded;
    }
}

Here is the call graph for this function:

OptionList & PLearn::CompactVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 77 of file CompactVMatrix.cc.

OptionMap & PLearn::CompactVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 77 of file CompactVMatrix.cc.

RemoteMethodMap & PLearn::CompactVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 77 of file CompactVMatrix.cc.

void PLearn::CompactVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 841 of file CompactVMatrix.cc.

References data, PLearn::deepCopyField(), fixedpoint_max, fixedpoint_min, n_symbol_values, and variables_permutation.

Here is the call graph for this function:

int PLearn::CompactVMatrix::nbits ( ) [inline]

Definition at line 80 of file CompactVMatrix.h.

{ return n_bits; }
int PLearn::CompactVMatrix::nfixedpoint ( ) [inline]

Definition at line 82 of file CompactVMatrix.h.

{ return n_fixedpoint; }
int PLearn::CompactVMatrix::nsymbols ( ) [inline]

Definition at line 81 of file CompactVMatrix.h.

{ return n_symbols; }
TVec<int>& PLearn::CompactVMatrix::permutation_vector ( ) [inline]

this vector is filled only when the CompactVMatrix was constructed from a VMat, and it provides the permutation of the original columns to order them into (bits, bytes, fixedpoint)

Definition at line 156 of file CompactVMatrix.h.

void PLearn::CompactVMatrix::perturb ( int  i,
Vec  row,
real  noise_level,
int  n_last 
)

create in the elements of row (except the n_last ones) a perturbed version of the i-th row of the database. This random perturbation is based on the unconditional statistics which should be present in the fieldstats; the noise level can be modulated with the noise_level argument (a value of 1 will perturb by as much as the noise seen in the unconditional statistics). Continuous variables are resampled around the current value with sigma = noise_leve * unconditional_sigma. Discrete variables are resampled with a distribution that is a mixture: (1-noise_level)*(probability mass on all current value)+noise_level*(unconditional distr)

Definition at line 613 of file CompactVMatrix.cc.

References b, PLearn::binomial_sample(), c, data, PLearn::Storage< T >::data, PLearn::TVec< T >::data(), delta, PLearn::VMatrix::fieldstats, fixedpoint_max, fixedpoint_min, fixedpoint_offset, j, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, PLearn::multinomial_sample(), n, n_bits, n_fixedpoint, n_symbol_values, n_symbols, n_variables, PLearn::normal_sample(), one_hot_encoding, PLERROR, PLearn::VMFieldStat::prob(), PLearn::TVec< T >::resize(), row_n_bytes, PLearn::TVec< T >::size(), symbols_offset, PLearn::short_and_twobytes::twobytes, u, PLearn::short_and_twobytes::us, PLearn::var(), PLearn::VMatrix::width(), and PLearn::VMatrix::width_.

{
#ifdef BOUNDCHECK
    if (i<0 || i>=length_)
        PLERROR("CompactVMatrix::perturb, row %d out of bounds [0,%d]",i,length_-1);
    if (v.length()!=width_)
        PLERROR("CompactVMatrix::perturb, length of v (%d) should be equal to width of VMat (%d)",v.length(),width());
#endif
    if (fieldstats.size()!=n_variables)
        PLERROR("CompactVMatrix::perturb: stats not computed or wrong size");
    if (noise_level<0 || noise_level>1)
        PLERROR("CompactVMatrix::perturb: noise_level=%g, should be in [0,1]",noise_level);

    unsigned char* encoded_row = &data.data[i*row_n_bytes];
    real* vp=v.data();
    int c=0;
    int var=0;
    Vec probs(width_);
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte=encoded_row[b];
        for (int j=0;j<8 && c<n_bits;j++,c++,var++)
        {
            int bit = byte & 1;
            byte >>= 1; // shift right once
            vp[c]=binomial_sample((1-noise_level)*bit+noise_level*fieldstats[var].prob(1));
        }
    }
    for (int b=0;b<n_symbols;b++,var++)
    {
        int byte = encoded_row[symbols_offset+b];
        int nv=n_symbol_values[b];
        probs.resize(nv);
        VMFieldStat& stat=fieldstats[var];
        for (int val=0;val<nv;val++)
            if (val==byte)
                probs[val]=(1-noise_level)+noise_level*stat.prob(val);
            else
                probs[val]=noise_level*stat.prob(val);
        byte = multinomial_sample(probs);
        if (one_hot_encoding)
        {
            int n=n_symbol_values[b];
            for (int j=0;j<n;j++) vp[c+j]=0;
            vp[c+byte]=1;
            c+=n;
        }
        else vp[c++]=byte;
    }
    unsigned char* fixed_point_numbers = &encoded_row[fixedpoint_offset];
    for (int j=0;j<n_fixedpoint;j++,c++,var++)
    {
        unsigned char *uc = &fixed_point_numbers[2*j];
        short_and_twobytes u;
        u.twobytes[0]=uc[0];
        u.twobytes[1]=uc[1];
        real decoded = u.us*delta[j]+fixedpoint_min[j];
        // correct rounding errors for integers, due to fixed-point low precision
        real rounded_decoded = rint(decoded);
        if (fabs(rounded_decoded-decoded)<1e-4)
            decoded = rounded_decoded;
        if (var<n_variables-n_last)
        {
            int ntry=0;
            do
            {
                vp[c]=decoded+noise_level*fieldstats[var].stddev()*normal_sample();
                ntry++;
                if (ntry>=100)
                    PLERROR("CompactVMatrix::perturb:Something wrong in resampling, tried 100 times");
            }
            while (vp[c]<fixedpoint_min[j] || vp[c]>fixedpoint_max[j]);
        }
        else
            vp[c]=decoded;
    }
}

Here is the call graph for this function:

void PLearn::CompactVMatrix::putRow ( int  i,
Vec  v 
) [virtual]

v is possibly one-hot-encoded (according to one_hot_encoding flag) and the variables are in the same order as for getRow.

Reimplemented from PLearn::VMatrix.

Definition at line 535 of file CompactVMatrix.cc.

References putSubRow().

Referenced by append(), and CompactVMatrix().

{
    putSubRow(i,0,v);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::CompactVMatrix::putSubRow ( int  i,
int  j,
Vec  v 
) [virtual]

It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead)

Reimplemented from PLearn::VMatrix.

Definition at line 540 of file CompactVMatrix.cc.

References b, c, PLearn::TVec< T >::data(), PLearn::Storage< T >::data, data, delta, PLearn::fast_exact_is_equal(), fixedpoint_min, fixedpoint_offset, PLearn::RowBufferedVMatrix::invalidateBuffer(), j, n, n_bits, n_fixedpoint, n_symbol_values, n_symbols, one_hot_encoding, PLERROR, row_n_bytes, and symbols_offset.

Referenced by putRow().

{
    unsigned char* encoded_row = &data.data[i*row_n_bytes];
    real* vp=v.data();
    int c=0;
    // 1 vector element ==> 1 bit
    for (int b=0;b<symbols_offset;b++)
    {
        unsigned char byte=0;
        for (int k=0;k<8 && c<n_bits;k++,c++)
            if (c>=j)
                byte |= int(vp[c-j]) << k; // shift to right bit position
        encoded_row[b]=byte;
    }
    // if (one_hot_encoding)
    //   n vector elements in one-hot-code ==> 1 byte
    // else
    //   1 vector element (integer between 0 and n-1) ==> 1 byte
    int n=0;
    if (one_hot_encoding)
        for (int b=0;b<n_symbols;b++,c+=n)
        {
            n=n_symbol_values[b];
            if (c>=j)
            {
                int pos=-1;
                for (int k=0;k<n;k++)
                {
                    real vk=vp[c+k-j];
                    if (!fast_exact_is_equal(vk, 0) &&
                        !fast_exact_is_equal(vk, 1))
                        PLERROR("CompactVMatrix::putRow(%d,v): v[%d]=%g!=0 or 1 (not one-hot-code)",
                                i,c,vk);
                    if (fast_exact_is_equal(vk, 1))
                    {
                        if (pos<0) pos=k;
                        else PLERROR("CompactVMatrix::putRow(%d,v): %d-th symbol not one-hot-encoded",
                                     i,b);
                    }
                }
                if (pos<0)
                    PLERROR("CompactVMatrix::putRow(%d,v): %d-th symbol not one-hot-encoded",
                            i,b);
                encoded_row[symbols_offset+b] = pos;
            }
        }
    else
        for (int b=0;b<n_symbols;b++,c++)
            if (c>=j)
            {
                real val = vp[c-j];
                int s = int(val);
                if (!fast_exact_is_equal(s, val))
                    PLERROR("CompactVMatrix::encodeAndPutRow(%d,v): v[%d]=%g not an integer",
                            i,c,val);
                encoded_row[symbols_offset+b] = s; // ASSUMES THAT v IS NOT ONE-HOT ENCODED
                if (s<0 || s>=n_symbol_values[b])
                    PLERROR("CompactVMatrix::encodeAndPutRow(%d,v): v[%d]=%d not in expected range (0,%d)",
                            i,c,s,n_symbol_values[b]-1);
            }

    // 1 vector element (real betweeen fixedpoint_min and fixedpoint_max) ==> 2 bytes
    //
    // WARNING: COULD THIS CAUSE PROBLEMS IF fixedpoint_offset IS NOT A MULTIPLE OF 4
    // ON SOME MACHINES?
    unsigned short* fixed_point_numbers = (unsigned short*)&encoded_row[fixedpoint_offset];
    for (int k=0;k<n_fixedpoint;k++,c++)
        if (c>=j)
            fixed_point_numbers[k]=(unsigned short)((vp[c-j]-fixedpoint_min[k])/delta[k]);

    invalidateBuffer();
}

Here is the call graph for this function:

Here is the caller graph for this function:

virtual void PLearn::CompactVMatrix::save ( const PPath filename) const [inline, virtual]

save everything in file (data and auxiliary information), binary format

calls write

Reimplemented from PLearn::VMatrix.

Definition at line 185 of file CompactVMatrix.h.

References PLearn::save().

Here is the call graph for this function:

void PLearn::CompactVMatrix::set_n_bits_in_byte ( ) [static, protected]

Definition at line 59 of file CompactVMatrix.cc.

References i, j, and n.

Referenced by CompactVMatrix().

{
    if (n_bits_in_byte[255]!=8)
    {
        for (int i=0;i<256;i++)
        {
            int n=0;
            unsigned char byte=i;
            for (int j=0;j<8;j++)
            {
                n += byte & 1;
                byte >>= 1;
            }
            n_bits_in_byte[i]=n;
        }
    }
}

Here is the caller graph for this function:

void PLearn::CompactVMatrix::setOneHotMode ( bool  on = true)

Definition at line 281 of file CompactVMatrix.cc.

References n_variables, normal_width, one_hot_encoding, and PLearn::VMatrix::width_.

Referenced by append(), and CompactVMatrix().

Here is the caller graph for this function:

real PLearn::CompactVMatrix::squareDifference ( int  i,
int  j 
) [virtual]

encoding (v is not one-hot, and the variables in v are in the "original" order

return the square difference between row i and row j, excluding n_last columns

Definition at line 488 of file CompactVMatrix.cc.

References dotProduct(), i, j, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, and row_norms.

{
    if (row_norms.length()==0)
        row_norms = Vec(length_,-1.0);
    real normi = row_norms[i];
    if (normi<0) normi=row_norms[i]=dotProduct(i,i);
    real normj = row_norms[j];
    if (normj<0) normj=row_norms[j]=dotProduct(j,j);
    return normi + normj - 2 * dotProduct(i,j);
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 192 of file CompactVMatrix.h.

Storage<unsigned char> PLearn::CompactVMatrix::data [protected]

Each row of the matrix holds in order: bits, 1-byte symbols, fixed point numbers.

Definition at line 66 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), makeDeepCopyFromShallowCopy(), perturb(), and putSubRow().

(fixedpoint_max-fixedpoint_min)/2^16

Definition at line 85 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

the ranges of each number for fixed point encoding

Definition at line 84 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), makeDeepCopyFromShallowCopy(), and perturb().

where in each row the fixed point numbers start

Definition at line 199 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

number of binary symbols per row

Definition at line 73 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

unsigned char PLearn::CompactVMatrix::n_bits_in_byte [static, protected]

CompactVMatrix.

and provides the permutation of the original columns in order to order them into (bits, bytes, fixedpoint) variables_permutation[new_column]=old_column (not in one-hot code)

Definition at line 91 of file CompactVMatrix.h.

Referenced by dot().

number of fixed point numbers per row

Definition at line 75 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

used by dotProduct and squareDifference to specify # of last columns to ignore

Definition at line 95 of file CompactVMatrix.h.

Referenced by CompactVMatrix(), dot(), and dotProduct().

for each 1-byte symbol, the number of possible values

Definition at line 79 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), makeDeepCopyFromShallowCopy(), perturb(), and putSubRow().

number of 1-byte symbols per row

Definition at line 74 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

= n_bits + n_symbols + n_fixedpoint

Definition at line 76 of file CompactVMatrix.h.

Referenced by CompactVMatrix(), perturb(), and setOneHotMode().

the value of width_ when one_hot_encoding=true

Definition at line 97 of file CompactVMatrix.h.

Referenced by CompactVMatrix(), and setOneHotMode().

the 1-byte symbols are converted to one-hot encoding by get

Definition at line 77 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), getNewRow(), perturb(), putSubRow(), and setOneHotMode().

# of bytes per row

Definition at line 72 of file CompactVMatrix.h.

Referenced by append(), CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

to cache the norms of the rows for squareDifference method

Definition at line 200 of file CompactVMatrix.h.

Referenced by squareDifference().

auxiliary

where in each row the symbols start

Definition at line 198 of file CompactVMatrix.h.

Referenced by CompactVMatrix(), dot(), encodeAndPutRow(), getNewRow(), perturb(), and putSubRow().

this variable is used only when constructed from VMat

Definition at line 86 of file CompactVMatrix.h.

Referenced by CompactVMatrix(), encodeAndPutRow(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines