Public Member Functions | Protected Member Functions

CKRR Class Reference

Detailed Description

Class KRR implements Kernel Ridge Regression - a regularized least square method for classification and regression.

It is similar to support vector machines (cf. CSVM). However in contrast to SVMs a different objective is optimized that leads to a dense solution (thus not only a few support vectors are active in the end but all training examples). This makes it only applicable to rather few (a couple of thousand) training examples. In case a linear kernel is used RR is closely related to Fishers Linear Discriminant (cf. LDA).

Internally (for linear kernels) it is solved via minimizing the following system

\[ \frac{1}{2}\left(\sum_{i=1}^N(y_i-{\bf w}\cdot {\bf x}_i)^2 + \tau||{\bf w}||^2\right) \]

which is boils down to solving a linear system

\[ {\bf w} = \left(\tau {\bf I}+ \sum_{i=1}^N{\bf x}_i{\bf x}_i^T\right)^{-1}\left(\sum_{i=1}^N y_i{\bf x}_i\right) \]

and in the kernel case

\[ {\bf \alpha}=\left({\bf K}+\tau{\bf I}\right)^{-1}{\bf y} \]

where K is the kernel matrix and y the vector of labels. The expressed solution can again be written as a linear combination of kernels (cf. CKernelMachine) with bias $b=0$.

Definition at line 54 of file KRR.h.

Inheritance diagram for CKRR:
Inheritance graph

List of all members.

Public Member Functions

 CKRR ()
 CKRR (float64_t tau, CKernel *k, CLabels *lab)
virtual ~CKRR ()
void set_tau (float64_t t)
virtual CLabelsapply ()
virtual float64_t apply (int32_t num)
virtual bool load (FILE *srcfile)
virtual bool save (FILE *dstfile)
virtual EClassifierType get_classifier_type ()
virtual const char * get_name () const

Protected Member Functions

virtual bool train_machine (CFeatures *data=NULL)

Constructor & Destructor Documentation

CKRR (  ) 

default constructor

Definition at line 21 of file KRR.cpp.

CKRR ( float64_t  tau,
CKernel k,
CLabels lab 


tau regularization constant tau
k kernel
lab labels

Definition at line 28 of file KRR.cpp.

~CKRR (  )  [virtual]

Definition at line 38 of file KRR.cpp.

Member Function Documentation

CLabels * apply (  )  [virtual]

classify regression

resulting labels

Reimplemented from CKernelMachine.

Definition at line 99 of file KRR.cpp.

float64_t apply ( int32_t  num  )  [virtual]

classify one example

num which example to classify

Reimplemented from CKernelMachine.

Definition at line 125 of file KRR.cpp.

virtual EClassifierType get_classifier_type (  )  [virtual]

get classifier type

classifier type KRR

Reimplemented from CMachine.

Definition at line 106 of file KRR.h.

virtual const char* get_name (  )  const [virtual]
object name

Reimplemented from CKernelMachine.

Definition at line 112 of file KRR.h.

bool load ( FILE *  srcfile  )  [virtual]

load regression from file

srcfile file to load from
if loading was successful

Reimplemented from CMachine.

Definition at line 85 of file KRR.cpp.

bool save ( FILE *  dstfile  )  [virtual]

save regression to file

dstfile file to save to
if saving was successful

Reimplemented from CMachine.

Definition at line 92 of file KRR.cpp.

void set_tau ( float64_t  t  ) 

set regularization constant

t new tau

Definition at line 73 of file KRR.h.

bool train_machine ( CFeatures data = NULL  )  [protected, virtual]

train regression

data training data (parameter can be avoided if distance or kernel-based regressors are used and distance/kernels are initialized with train data)
whether training was successful

Reimplemented from CMachine.

Definition at line 43 of file KRR.cpp.

The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

SHOGUN Machine Learning Toolbox - Documentation