Public Member Functions | Protected Member Functions | Static Protected Member Functions | Protected Attributes

CSVRLight Class Reference

Detailed Description

Class SVRLight, performs support vector regression using SVMLight.

The SVR solution can be expressed as

\[ f({\bf x})=\sum_{i=1}^{N} \alpha_i k({\bf x}, {\bf x_i})+b \]

where $\alpha$ and $b$ are determined in training, i.e. using a pre-specified kernel, a given tube-epsilon for the epsilon insensitive loss, the follwoing quadratic problem is minimized (using the chunking decomposition technique)

\begin{eqnarray*} \max_{{\bf \alpha},{\bf \alpha}^*} &-\frac{1}{2}\sum_{i,j=1}^N(\alpha_i-\alpha_i^*)(\alpha_j-\alpha_j^*){\bf x}_i^T {\bf x}_j -\sum_{i=1}^N(\alpha_i+\alpha_i^*)\epsilon - \sum_{i=1}^N(\alpha_i-\alpha_i^*)y_i\\ \mbox{wrt}:& {\bf \alpha},{\bf \alpha}^*\in{\bf R}^N\\ \mbox{s.t.}:& 0\leq \alpha_i,\alpha_i^*\leq C,\, \forall i=1\dots N\\ &\sum_{i=1}^N(\alpha_i-\alpha_i^*)y_i=0 \end{eqnarray*}

Note that the SV regression problem is reduced to the standard SV classification problem by introducing artificial labels $-y_i$ which leads to the epsilon insensitive loss constraints *

\begin{eqnarray*} {\bf w}^T{\bf x}_i+b-c_i-\xi_i\leq 0,&\, \forall i=1\dots N\\ -{\bf w}^T{\bf x}_i-b-c_i^*-\xi_i^*\leq 0,&\, \forall i=1\dots N \end{eqnarray*}

with $c_i=y_i+ \epsilon$ and $c_i^*=-y_i+ \epsilon$

This implementation supports multiple kernel learning, i.e. if a CCombinedKernel is used the weights $\beta$ in $ k_{combined}({\bf x}, {\bf x'}) = \sum_{m=0}^M \beta_m k_m({\bf x}, {\bf x'})$ can be determined in training (cf. Large Scale Multiple Kernel Learning Sonnenburg, Raetsch, Schaefer, Schoelkopf 2006).

linadd optimizations were implemented for kernels that support it (most string kernels and the linear kernel), which will result in significant speedups.

Definition at line 62 of file SVRLight.h.

Inheritance diagram for CSVRLight:
Inheritance graph
[legend]

List of all members.

Public Member Functions

 CSVRLight ()
 CSVRLight (float64_t C, float64_t epsilon, CKernel *k, CLabels *lab)
virtual ~CSVRLight ()
virtual EClassifierType get_classifier_type ()
void svr_learn ()
virtual float64_t compute_objective_function (float64_t *a, float64_t *lin, float64_t *c, float64_t *eps, int32_t *label, int32_t totdoc)
virtual void update_linear_component (int32_t *docs, int32_t *label, int32_t *active2dnum, float64_t *a, float64_t *a_old, int32_t *working2dnum, int32_t totdoc, float64_t *lin, float64_t *aicache, float64_t *c)
virtual void update_linear_component_mkl (int32_t *docs, int32_t *label, int32_t *active2dnum, float64_t *a, float64_t *a_old, int32_t *working2dnum, int32_t totdoc, float64_t *lin, float64_t *aicache, float64_t *c)
virtual void update_linear_component_mkl_linadd (int32_t *docs, int32_t *label, int32_t *active2dnum, float64_t *a, float64_t *a_old, int32_t *working2dnum, int32_t totdoc, float64_t *lin, float64_t *aicache, float64_t *c)
void call_mkl_callback (float64_t *a, int32_t *label, float64_t *lin, float64_t *c, int32_t totdoc)
virtual void reactivate_inactive_examples (int32_t *label, float64_t *a, SHRINK_STATE *shrink_state, float64_t *lin, float64_t *c, int32_t totdoc, int32_t iteration, int32_t *inconsistent, int32_t *docs, float64_t *aicache, float64_t *maxdiff)
virtual const char * get_name () const

Protected Member Functions

int32_t regression_fix_index (int32_t i)
virtual float64_t compute_kernel (int32_t i, int32_t j)
virtual bool train_machine (CFeatures *data=NULL)

Static Protected Member Functions

static void * update_linear_component_linadd_helper (void *params)
static int32_t regression_fix_index2 (int32_t i, int32_t num_vectors)

Protected Attributes

int32_t num_vectors

Constructor & Destructor Documentation

CSVRLight (  ) 

default constructor

Definition at line 57 of file SVRLight.cpp.

CSVRLight ( float64_t  C,
float64_t  epsilon,
CKernel *  k,
CLabels *  lab 
)

constructor

Parameters:
C constant C
epsilon epsilon
k kernel
lab labels

Definition at line 51 of file SVRLight.cpp.

virtual ~CSVRLight (  )  [virtual]

default destructor

Definition at line 78 of file SVRLight.h.

Member Function Documentation

void call_mkl_callback ( float64_t *  a,
int32_t *  label,
float64_t *  lin,
float64_t *  c,
int32_t  totdoc 
)

call mkl callback

Parameters:
a 
label 
lin 
c 
totdoc 

Definition at line 582 of file SVRLight.cpp.

virtual float64_t compute_kernel ( int32_t  i,
int32_t  j 
) [protected, virtual]

compute kernel at given index

Parameters:
i index i
j index j
Returns:
kernel value at i,j

Reimplemented from CSVMLight.

Definition at line 233 of file SVRLight.h.

float64_t compute_objective_function ( float64_t *  a,
float64_t *  lin,
float64_t *  c,
float64_t *  eps,
int32_t *  label,
int32_t  totdoc 
) [virtual]

compute objective function

Parameters:
a a
lin lin
c c
eps eps
label label
totdoc totdoc

Reimplemented from CSVMLight.

Definition at line 327 of file SVRLight.cpp.

virtual EClassifierType get_classifier_type (  )  [virtual]

get classifier type

Returns:
classifier type SVRLIGHT

Reimplemented from CSVMLight.

Definition at line 84 of file SVRLight.h.

virtual const char* get_name ( void   )  const [virtual]
Returns:
object name

Reimplemented from CSVMLight.

Definition at line 190 of file SVRLight.h.

void reactivate_inactive_examples ( int32_t *  label,
float64_t *  a,
SHRINK_STATE *  shrink_state,
float64_t *  lin,
float64_t *  c,
int32_t  totdoc,
int32_t  iteration,
int32_t *  inconsistent,
int32_t *  docs,
float64_t *  aicache,
float64_t *  maxdiff 
) [virtual]

reactivate inactive examples

Parameters:
label label
a a
shrink_state shrink state
lin lin
c c
totdoc totdoc
iteration iteration
inconsistent inconsistent
docs docs
aicache ai cache
maxdiff maxdiff

Reimplemented from CSVMLight.

Definition at line 637 of file SVRLight.cpp.

int32_t regression_fix_index ( int32_t  i  )  [protected]

regression fix index

Parameters:
i i
Returns:
fix index

Definition at line 204 of file SVRLight.h.

static int32_t regression_fix_index2 ( int32_t  i,
int32_t  num_vectors 
) [static, protected]

regression fix index2

Parameters:
i i
num_vectors number of vectors
Returns:
fix index

Definition at line 218 of file SVRLight.h.

void svr_learn (  ) 

SVR learn

Definition at line 144 of file SVRLight.cpp.

bool train_machine ( CFeatures *  data = NULL  )  [protected, virtual]

train regression

Parameters:
data training data (parameter can be avoided if distance or kernel-based regressors are used and distance/kernels are initialized with train data)
Returns:
whether training was successful

Reimplemented from CSVMLight.

Definition at line 62 of file SVRLight.cpp.

void update_linear_component ( int32_t *  docs,
int32_t *  label,
int32_t *  active2dnum,
float64_t *  a,
float64_t *  a_old,
int32_t *  working2dnum,
int32_t  totdoc,
float64_t *  lin,
float64_t *  aicache,
float64_t *  c 
) [virtual]

update linear component

Parameters:
docs docs
label label
active2dnum active2dnum
a a
a_old a old
working2dnum working2dnum
totdoc totdoc
lin lin
aicache ai cache
c c

Reimplemented from CSVMLight.

Definition at line 364 of file SVRLight.cpp.

void * update_linear_component_linadd_helper ( void *  params  )  [static, protected]

thread helper for update linear component linadd

Parameters:
params 

Reimplemented from CSVMLight.

Definition at line 351 of file SVRLight.cpp.

void update_linear_component_mkl ( int32_t *  docs,
int32_t *  label,
int32_t *  active2dnum,
float64_t *  a,
float64_t *  a_old,
int32_t *  working2dnum,
int32_t  totdoc,
float64_t *  lin,
float64_t *  aicache,
float64_t *  c 
) [virtual]

update linear component MKL

Parameters:
docs docs
label label
active2dnum active2dnum
a a
a_old a old
working2dnum working2dnum
totdoc totdoc
lin lin
aicache ai cache
c c

Definition at line 462 of file SVRLight.cpp.

void update_linear_component_mkl_linadd ( int32_t *  docs,
int32_t *  label,
int32_t *  active2dnum,
float64_t *  a,
float64_t *  a_old,
int32_t *  working2dnum,
int32_t  totdoc,
float64_t *  lin,
float64_t *  aicache,
float64_t *  c 
) [virtual]

update linear component MKL linadd

Parameters:
docs docs
label label
active2dnum active2dnum
a a
a_old a old
working2dnum working2dnum
totdoc totdoc
lin lin
aicache ai cache
c c

Definition at line 535 of file SVRLight.cpp.

Member Data Documentation

int32_t num_vectors [protected]

number of train elements

Definition at line 251 of file SVRLight.h.

The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines
SHOGUN Machine Learning Toolbox - Documentation