SHOGUN  v1.1.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
List of all members | Public Member Functions | Protected Member Functions
CKRR Class Reference

Detailed Description

Class KRR implements Kernel Ridge Regression - a regularized least square method for classification and regression.

It is similar to support vector machines (cf. CSVM). However in contrast to SVMs a different objective is optimized that leads to a dense solution (thus not only a few support vectors are active in the end but all training examples). This makes it only applicable to rather few (a couple of thousand) training examples. In case a linear kernel is used RR is closely related to Fishers Linear Discriminant (cf. LDA).

Internally (for linear kernels) it is solved via minimizing the following system

\[ \frac{1}{2}\left(\sum_{i=1}^N(y_i-{\bf w}\cdot {\bf x}_i)^2 + \tau||{\bf w}||^2\right) \]

which is boils down to solving a linear system

\[ {\bf w} = \left(\tau {\bf I}+ \sum_{i=1}^N{\bf x}_i{\bf x}_i^T\right)^{-1}\left(\sum_{i=1}^N y_i{\bf x}_i\right) \]

and in the kernel case

\[ {\bf \alpha}=\left({\bf K}+\tau{\bf I}\right)^{-1}{\bf y} \]

where K is the kernel matrix and y the vector of labels. The expressed solution can again be written as a linear combination of kernels (cf. CKernelMachine) with bias $b=0$.

Definition at line 54 of file KRR.h.

Inheritance diagram for CKRR:
Inheritance graph
[legend]

Public Member Functions

 CKRR ()
 CKRR (float64_t tau, CKernel *k, CLabels *lab)
virtual ~CKRR ()
void set_tau (float64_t t)
virtual CLabelsapply ()
virtual float64_t apply (int32_t num)
virtual bool load (FILE *srcfile)
virtual bool save (FILE *dstfile)
virtual EClassifierType get_classifier_type ()
virtual const char * get_name () const
- Public Member Functions inherited from CKernelMachine
 CKernelMachine ()
virtual ~CKernelMachine ()
void set_kernel (CKernel *k)
CKernelget_kernel ()
void set_batch_computation_enabled (bool enable)
bool get_batch_computation_enabled ()
void set_linadd_enabled (bool enable)
bool get_linadd_enabled ()
void set_bias_enabled (bool enable_bias)
bool get_bias_enabled ()
float64_t get_bias ()
void set_bias (float64_t bias)
int32_t get_support_vector (int32_t idx)
float64_t get_alpha (int32_t idx)
bool set_support_vector (int32_t idx, int32_t val)
bool set_alpha (int32_t idx, float64_t val)
int32_t get_num_support_vectors ()
void set_alphas (SGVector< float64_t > alphas)
void set_support_vectors (SGVector< int32_t > svs)
SGVector< int32_t > get_support_vectors ()
SGVector< float64_tget_alphas ()
bool create_new_model (int32_t num)
bool init_kernel_optimization ()
virtual CLabelsapply (CFeatures *data)
- Public Member Functions inherited from CMachine
 CMachine ()
virtual ~CMachine ()
virtual bool train (CFeatures *data=NULL)
virtual void set_labels (CLabels *lab)
virtual CLabelsget_labels ()
virtual float64_t get_label (int32_t i)
void set_max_train_time (float64_t t)
float64_t get_max_train_time ()
void set_solver_type (ESolverType st)
ESolverType get_solver_type ()
virtual void set_store_model_features (bool store_model)
- Public Member Functions inherited from CSGObject
 CSGObject ()
 CSGObject (const CSGObject &orig)
virtual ~CSGObject ()
virtual bool is_generic (EPrimitiveType *generic) const
template<class T >
void set_generic ()
void unset_generic ()
virtual void print_serializable (const char *prefix="")
virtual bool save_serializable (CSerializableFile *file, const char *prefix="")
virtual bool load_serializable (CSerializableFile *file, const char *prefix="")
void set_global_io (SGIO *io)
SGIOget_global_io ()
void set_global_parallel (Parallel *parallel)
Parallelget_global_parallel ()
void set_global_version (Version *version)
Versionget_global_version ()
SGVector< char * > get_modelsel_names ()
char * get_modsel_param_descr (const char *param_name)
index_t get_modsel_param_index (const char *param_name)

Protected Member Functions

virtual bool train_machine (CFeatures *data=NULL)
- Protected Member Functions inherited from CKernelMachine
virtual void store_model_features ()

Additional Inherited Members

- Static Public Member Functions inherited from CKernelMachine
static void * apply_helper (void *p)
- Public Attributes inherited from CSGObject
SGIOio
Parallelparallel
Versionversion
Parameterm_parameters
Parameterm_model_selection_parameters
- Protected Attributes inherited from CKernelMachine
CKernelkernel
bool use_batch_computation
bool use_linadd
bool use_bias
float64_t m_bias
SGVector< float64_tm_alpha
SGVector< int32_t > m_svs

Constructor & Destructor Documentation

CKRR ( )

default constructor

Definition at line 21 of file KRR.cpp.

CKRR ( float64_t  tau,
CKernel k,
CLabels lab 
)

constructor

Parameters
tauregularization constant tau
kkernel
lablabels

Definition at line 28 of file KRR.cpp.

~CKRR ( )
virtual

Definition at line 38 of file KRR.cpp.

Member Function Documentation

CLabels * apply ( )
virtual

classify regression

Returns
resulting labels

Reimplemented from CKernelMachine.

Definition at line 99 of file KRR.cpp.

float64_t apply ( int32_t  num)
virtual

classify one example

Parameters
numwhich example to classify
Returns
result

Reimplemented from CKernelMachine.

Definition at line 125 of file KRR.cpp.

virtual EClassifierType get_classifier_type ( )
virtual

get classifier type

Returns
classifier type KRR

Reimplemented from CMachine.

Definition at line 106 of file KRR.h.

virtual const char* get_name ( ) const
virtual
Returns
object name

Reimplemented from CKernelMachine.

Definition at line 112 of file KRR.h.

bool load ( FILE *  srcfile)
virtual

load regression from file

Parameters
srcfilefile to load from
Returns
if loading was successful

Reimplemented from CMachine.

Definition at line 85 of file KRR.cpp.

bool save ( FILE *  dstfile)
virtual

save regression to file

Parameters
dstfilefile to save to
Returns
if saving was successful

Reimplemented from CMachine.

Definition at line 92 of file KRR.cpp.

void set_tau ( float64_t  t)

set regularization constant

Parameters
tnew tau

Definition at line 73 of file KRR.h.

bool train_machine ( CFeatures data = NULL)
protectedvirtual

train regression

Parameters
datatraining data (parameter can be avoided if distance or kernel-based regressors are used and distance/kernels are initialized with train data)
Returns
whether training was successful

Reimplemented from CMachine.

Definition at line 43 of file KRR.cpp.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation