PLearn 0.1
|
Linear kernel that can be used for Automatic Relevance Determination. More...
#include <LinearARDKernel.h>
Public Member Functions | |
LinearARDKernel () | |
Default constructor. | |
virtual real | evaluate (const Vec &x1, const Vec &x2) const |
Compute K(x1,x2). | |
virtual void | computeGramMatrix (Mat K) const |
Compute the Gram Matrix. | |
virtual void | computeGramMatrixDerivative (Mat &KD, const string &kernel_param, real epsilon=1e-6) const |
Directly compute the derivative with respect to hyperparameters (Faster than finite differences...) | |
virtual void | evaluate_all_i_x (const Vec &x, const Vec &k_xi_x, real squared_norm_of_x=-1, int istart=0) const |
Fill k_xi_x with K(x_i, x), for all i from istart to istart + k_xi_x.length() - 1. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual LinearARDKernel * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Post-constructor. | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Protected Member Functions | |
real | derivIspSignalSigma (int i, int j, int arg, real K) const |
Derivative function with respect to isp_signal_sigma. | |
real | derivIspGlobalSigma (int i, int j, int arg, real K) const |
Derivative function with respect to isp_global_sigma. | |
void | computeGramMatrixDerivIspSignalSigma (Mat &KD) const |
void | computeGramMatrixDerivIspInputSigma (Mat &KD, int arg) const |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Private Types | |
typedef ARDBaseKernel | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. |
Linear kernel that can be used for Automatic Relevance Determination.
This is a simple linear (dot-product) kernel that provides a different length-scale parameter for each input variable. When used in conjunction with GaussianProcessRegressor it yields a Bayesian linear regression model with a non-isotropic prior. (It is not a particularly efficient way of performing linear regression, but can be useful as a benchmark against other kernels).
This kernel function is specified as:
k(x,y) = sf * (sum_i x_i * y_i / w_i) * k_kron(x,y)
where sf is softplus(isp_signal_sigma), w_i is softplus(isp_global_sigma + isp_input_sigma[i]), and k_kron(x,y) is the result of the KroneckerBaseKernel evaluation, or 1.0 if there are no Kronecker terms. Note that since the Kronecker terms are incorporated multiplicatively, the very presence of the term associated to this kernel can be gated by the value of some input variable(s) (that are incorporated within one or more Kronecker terms).
For best results, especially with moderately noisy data, IT IS IMPERATIVE to use whis kernel within a SummationKernel in conjunction with an IIDNoiseKernel, as follows (e.g. within a GaussianProcessRegressor):
kernel = SummationKernel(terms = [ LinearARDKernel(), IIDNoiseKernel() ] )
Note that to make its operations more robust when used with unconstrained optimization of hyperparameters, all hyperparameters of this kernel are specified in the inverse softplus domain. See IIDNoiseKernel for more explanations.
Definition at line 81 of file LinearARDKernel.h.
typedef ARDBaseKernel PLearn::LinearARDKernel::inherited [private] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 83 of file LinearARDKernel.h.
PLearn::LinearARDKernel::LinearARDKernel | ( | ) |
string PLearn::LinearARDKernel::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
OptionList & PLearn::LinearARDKernel::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
RemoteMethodMap & PLearn::LinearARDKernel::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
Object * PLearn::LinearARDKernel::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
StaticInitializer LinearARDKernel::_static_initializer_ & PLearn::LinearARDKernel::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
void PLearn::LinearARDKernel::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 96 of file LinearARDKernel.cc.
{ // ### Nothing to add here, simply calls build_ inherited::build(); build_(); }
void PLearn::LinearARDKernel::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 106 of file LinearARDKernel.cc.
{ // Ensure that we multiply in Kronecker terms inherited::m_default_value = 1.0; }
string PLearn::LinearARDKernel::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
void PLearn::LinearARDKernel::computeGramMatrix | ( | Mat | K | ) | const [virtual] |
Compute the Gram Matrix.
Reimplemented from PLearn::KroneckerBaseKernel.
Definition at line 145 of file LinearARDKernel.cc.
References i, j, m, PLearn::TMat< T >::mod(), n, PLASSERT, PLearn::TMat< T >::size(), and PLearn::softplus().
{ PLASSERT( !m_isp_input_sigma.size() || dataInputsize() == m_isp_input_sigma.size() ); PLASSERT( K.size() == 0 || m_data_cache.size() > 0 ); // Ensure data cached OK // Compute Kronecker gram matrix inherited::computeGramMatrix(K); // Precompute some terms. Make sure that the input sigmas don't get too // small real sf = softplus(m_isp_signal_sigma); m_input_sigma.resize(dataInputsize()); softplusFloor(m_isp_global_sigma, 1e-6); m_input_sigma.fill(m_isp_global_sigma); // Still in ISP domain for (int i=0, n=m_input_sigma.size() ; i<n ; ++i) { if (m_isp_input_sigma.size() > 0) { softplusFloor(m_isp_input_sigma[i], 1e-6); m_input_sigma[i] += m_isp_input_sigma[i]; } m_input_sigma[i] = softplus(m_input_sigma[i]); } // Compute Gram Matrix int l = data->length(); int m = K.mod(); int n = dataInputsize(); int cache_mod = m_data_cache.mod(); real *data_start = &m_data_cache(0,0); real *Ki = K[0]; // Start of current row real *Kij; // Current element along row real *input_sigma_data = m_input_sigma.data(); real *xi = data_start; for (int i=0 ; i<l ; ++i, xi += cache_mod, Ki+=m) { Kij = Ki; real *xj = data_start; for (int j=0; j<=i; ++j, xj += cache_mod) { // Kernel evaluation per se real *x1 = xi; real *x2 = xj; real *p_inpsigma = input_sigma_data; real the_dot = 0.0; int k = n; // Use Duff's device to unroll the following loop: // while (k--) { // the_dot += (*x1++ * *x2++) / *p_inpsigma++; // } switch (k % 8) { case 0: do { the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 7: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 6: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 5: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 4: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 3: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 2: the_dot += (*x1++ * *x2++) / *p_inpsigma++; case 1: the_dot += (*x1++ * *x2++) / *p_inpsigma++; } while((k -= 8) > 0); } // Multiplicatively update kernel matrix (already pre-filled with // Kronecker terms, or 1.0 if no Kronecker terms, as per build_). real Kij_cur = *Kij * sf * the_dot; *Kij++ = Kij_cur; } } if (cache_gram_matrix) { gram_matrix.resize(l,l); gram_matrix << K; gram_matrix_is_cached = true; } }
void PLearn::LinearARDKernel::computeGramMatrixDerivative | ( | Mat & | KD, |
const string & | kernel_param, | ||
real | epsilon = 1e-6 |
||
) | const [virtual] |
Directly compute the derivative with respect to hyperparameters (Faster than finite differences...)
Reimplemented from PLearn::Kernel.
Definition at line 224 of file LinearARDKernel.cc.
References derivIspGlobalSigma(), PLASSERT, PLearn::string_begins_with(), and PLearn::tolong().
{ static const string ISS("isp_signal_sigma"); static const string IGS("isp_global_sigma"); static const string IIS("isp_input_sigma["); if (kernel_param == ISS) { computeGramMatrixDerivIspSignalSigma(KD); // computeGramMatrixDerivNV< // LinearARDKernel, // &LinearARDKernel::derivIspSignalSigma>(KD, this, -1); } else if (kernel_param == IGS) { computeGramMatrixDerivNV< LinearARDKernel, &LinearARDKernel::derivIspGlobalSigma>(KD, this, -1); } else if (string_begins_with(kernel_param, IIS) && kernel_param[kernel_param.size()-1] == ']') { int arg = tolong(kernel_param.substr( IIS.size(), kernel_param.size() - IIS.size() - 1)); PLASSERT( arg < m_isp_input_sigma.size() ); computeGramMatrixDerivIspInputSigma(KD, arg); } else inherited::computeGramMatrixDerivative(KD, kernel_param, epsilon); }
void PLearn::LinearARDKernel::computeGramMatrixDerivIspInputSigma | ( | Mat & | KD, |
int | arg | ||
) | const [protected] |
Definition at line 306 of file LinearARDKernel.cc.
References PLearn::TMat< T >::data(), i, j, PLearn::TMat< T >::mod(), PLASSERT_MSG, PLearn::TMat< T >::resize(), PLearn::sigmoid(), and PLearn::softplus().
{ // Precompute some terms real signal_sigma = softplus(m_isp_signal_sigma); real input_sigma_arg = m_input_sigma[arg]; real input_sigma_sq = input_sigma_arg * input_sigma_arg; real input_sigmoid = sigmoid(m_isp_global_sigma + m_isp_input_sigma[arg]); // Compute Gram Matrix derivative w.r.t. isp_input_sigma[arg] int l = data->length(); PLASSERT_MSG( gram_matrix.width() == l && gram_matrix.length() == l, "To compute the derivative with respect to 'isp_input_sigma[i]', the\n" "Gram matrix must be precomputed and cached in LinearARDKernel."); // Variables that walk over the data matrix int cache_mod = m_data_cache.mod(); real *data_start = &m_data_cache(0,0); real *xi = data_start+arg; // Iterator on data rows // Variables that walk over the kernel derivative matrix (KD) KD.resize(l,l); real* KDi = KD.data(); // Start of row i real* KDij; // Current element on row i int KD_mod = KD.mod(); // Iterate on rows of derivative matrix for (int i=0 ; i<l ; ++i, xi += cache_mod, KDi += KD_mod) { KDij = KDi; real *xj = data_start+arg; // Inner iterator on data rows // Iterate on columns of derivative matrix for (int j=0 ; j <= i ; ++j, xj += cache_mod) { // Set into derivative matrix *KDij++ = - signal_sigma * (*xi * *xj) * input_sigmoid / input_sigma_sq; } } }
void PLearn::LinearARDKernel::computeGramMatrixDerivIspSignalSigma | ( | Mat & | KD | ) | const [protected] |
Definition at line 290 of file LinearARDKernel.cc.
References PLASSERT_MSG, PLearn::TMat< T >::resize(), PLearn::sigmoid(), and PLearn::softplus().
{ int l = data->length(); KD.resize(l,l); PLASSERT_MSG( gram_matrix.width() == l && gram_matrix.length() == l, "To compute the derivative with respect to 'isp_signal_sigma', the\n" "Gram matrix must be precomputed and cached in LinearARDKernel."); KD << gram_matrix; KD *= sigmoid(m_isp_signal_sigma)/softplus(m_isp_signal_sigma); }
void PLearn::LinearARDKernel::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 87 of file LinearARDKernel.cc.
{ // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::LinearARDKernel::declaringFile | ( | ) | [inline, static] |
LinearARDKernel * PLearn::LinearARDKernel::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
real PLearn::LinearARDKernel::derivIspGlobalSigma | ( | int | i, |
int | j, | ||
int | arg, | ||
real | K | ||
) | const [protected] |
Derivative function with respect to isp_global_sigma.
Definition at line 279 of file LinearARDKernel.cc.
References PLearn::fast_is_equal(), PLearn::sigmoid(), and PLearn::softplus().
Referenced by computeGramMatrixDerivative().
{ if (fast_is_equal(K,0.)) return 0.; return - K * sigmoid(m_isp_global_sigma) / softplus(m_isp_global_sigma); }
real PLearn::LinearARDKernel::derivIspSignalSigma | ( | int | i, |
int | j, | ||
int | arg, | ||
real | K | ||
) | const [protected] |
Derivative function with respect to isp_signal_sigma.
Definition at line 270 of file LinearARDKernel.cc.
References PLearn::sigmoid(), and PLearn::softplus().
{ // (No longer used; see computeGramMatrixDerivIspInputSigma below) return K*sigmoid(m_isp_signal_sigma)/softplus(m_isp_signal_sigma); }
Compute K(x1,x2).
Reimplemented from PLearn::KroneckerBaseKernel.
Definition at line 115 of file LinearARDKernel.cc.
References PLearn::TVec< T >::data(), PLearn::dot(), PLearn::fast_is_equal(), i, n, PLASSERT, PLearn::TVec< T >::size(), and PLearn::softplus().
{ PLASSERT( x1.size() == x2.size() ); PLASSERT( !m_isp_input_sigma.size() || x1.size() == m_isp_input_sigma.size() ); real gating_term = inherited::evaluate(x1,x2); if (fast_is_equal(gating_term, 0.0) || x1.size() == 0) return 0.0; real the_dot = 0.0; if (m_isp_input_sigma.size() > 0) { const real* px1 = x1.data(); const real* px2 = x2.data(); const real* pinpsig = m_isp_input_sigma.data(); for (int i=0, n=x1.size() ; i<n ; ++i) { the_dot += (*px1++ * *px2++) / softplus(m_isp_global_sigma + *pinpsig++); } } else { real global_sigma = softplus(m_isp_global_sigma); the_dot = dot(x1, x2) / global_sigma; } // Gate by Kronecker term return softplus(m_isp_signal_sigma) * the_dot * gating_term; }
void PLearn::LinearARDKernel::evaluate_all_i_x | ( | const Vec & | x, |
const Vec & | k_xi_x, | ||
real | squared_norm_of_x = -1 , |
||
int | istart = 0 |
||
) | const [virtual] |
Fill k_xi_x with K(x_i, x), for all i from istart to istart + k_xi_x.length() - 1.
Reimplemented from PLearn::Kernel.
Definition at line 260 of file LinearARDKernel.cc.
References x.
OptionList & PLearn::LinearARDKernel::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
OptionMap & PLearn::LinearARDKernel::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
RemoteMethodMap & PLearn::LinearARDKernel::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 78 of file LinearARDKernel.cc.
void PLearn::LinearARDKernel::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 350 of file LinearARDKernel.cc.
{ inherited::makeDeepCopyFromShallowCopy(copies); }
Reimplemented from PLearn::ARDBaseKernel.
Definition at line 118 of file LinearARDKernel.h.