SHOGUN  4.1.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
List of all members | Public Member Functions | Protected Member Functions | Protected Attributes
LBFGSMinimizer Class Reference

Detailed Description

The class wraps the Shogun's C-style LBFGS minimizer.

Definition at line 49 of file LBFGSMinimizer.h.

Inheritance diagram for LBFGSMinimizer:
Inheritance graph
[legend]

Public Member Functions

 LBFGSMinimizer ()
 
 LBFGSMinimizer (FirstOrderCostFunction *fun)
 
virtual ~LBFGSMinimizer ()
 
virtual float64_t minimize ()
 
virtual bool supports_batch_update () const
 
virtual void set_lbfgs_parameters (int m=100, int max_linesearch=1000, ELBFGSLineSearch linesearch=BACKTRACKING_STRONG_WOLFE, int max_iterations=1000, float64_t delta=0.0, int past=0, float64_t epsilon=1e-5, float64_t min_step=1e-20, float64_t max_step=1e+20, float64_t ftol=1e-4, float64_t wolfe=0.9, float64_t gtol=0.9, float64_t xtol=1e-16, float64_t orthantwise_c=0.0, int orthantwise_start=0, int orthantwise_end=1)
 
virtual CMinimizerContextsave_to_context ()
 
virtual void load_from_context (CMinimizerContext *context)
 
virtual void set_cost_function (FirstOrderCostFunction *fun)
 
virtual void set_penalty_weight (float64_t penalty_weight)
 
virtual void set_penalty_type (Penalty *penalty_type)
 

Protected Member Functions

virtual void init_minimization ()
 
virtual void update_context (CMinimizerContext *context)
 
virtual float64_t get_penalty (SGVector< float64_t > var)
 
virtual void update_gradient (SGVector< float64_t > gradient, SGVector< float64_t > var)
 

Protected Attributes

int m_m
 
int m_max_linesearch
 
int m_linesearch
 
int m_max_iterations
 
float64_t m_delta
 
int m_past
 
float64_t m_epsilon
 
float64_t m_min_step
 
float64_t m_max_step
 
float64_t m_ftol
 
float64_t m_wolfe
 
float64_t m_gtol
 
float64_t m_xtol
 
float64_t m_orthantwise_c
 
int m_orthantwise_start
 
int m_orthantwise_end
 
SGVector< float64_tm_target_variable
 
FirstOrderCostFunctionm_fun
 
Penaltym_penalty_type
 
float64_t m_penalty_weight
 

Constructor & Destructor Documentation

Definition at line 37 of file LBFGSMinimizer.cpp.

Constructor

Parameters
funcost function (user have to manully delete the pointer)

Definition at line 47 of file LBFGSMinimizer.cpp.

~LBFGSMinimizer ( )
virtual

Definition at line 43 of file LBFGSMinimizer.cpp.

Member Function Documentation

virtual float64_t get_penalty ( SGVector< float64_t var)
protectedvirtualinherited

Get the penalty given target variables For L2 penalty, the target variable is \(w\) and the value of penalty is \(\lambda \frac{w^t w}{2}\), where \(\lambda\) is the weight of penalty

Parameters
varthe variable used in regularization

Definition at line 164 of file FirstOrderMinimizer.h.

void init_minimization ( )
protectedvirtual

Init before minimization

Definition at line 94 of file LBFGSMinimizer.cpp.

virtual void load_from_context ( CMinimizerContext context)
virtual

Load the given context object to restores mutable variables Usually it is used in deserialization.

Parameters
contexta context object

Reimplemented from FirstOrderMinimizer.

Definition at line 142 of file LBFGSMinimizer.h.

float64_t minimize ( )
virtual

Do minimization and get the optimal value

Returns
optimal value

Implements FirstOrderMinimizer.

Definition at line 101 of file LBFGSMinimizer.cpp.

virtual CMinimizerContext* save_to_context ( )
virtual

Return a context object which stores mutable variables Usually it is used in serialization.

Returns
a context object

Reimplemented from FirstOrderMinimizer.

Definition at line 135 of file LBFGSMinimizer.h.

virtual void set_cost_function ( FirstOrderCostFunction fun)
virtualinherited

Set cost function used in the minimizer

Parameters
funthe cost function

Definition at line 92 of file FirstOrderMinimizer.h.

void set_lbfgs_parameters ( int  m = 100,
int  max_linesearch = 1000,
ELBFGSLineSearch  linesearch = BACKTRACKING_STRONG_WOLFE,
int  max_iterations = 1000,
float64_t  delta = 0.0,
int  past = 0,
float64_t  epsilon = 1e-5,
float64_t  min_step = 1e-20,
float64_t  max_step = 1e+20,
float64_t  ftol = 1e-4,
float64_t  wolfe = 0.9,
float64_t  gtol = 0.9,
float64_t  xtol = 1e-16,
float64_t  orthantwise_c = 0.0,
int  orthantwise_start = 0,
int  orthantwise_end = 1 
)
virtual

set L-BFGS parameters For details please see shogun/optimization/lbfgs/lbfgs.h

Parameters
mThe number of corrections to approximate the inverse hessian matrix. Default value is 100.
max_linesearchThe maximum number of trials to do line search for each L-BFGS update. Default value is 1000.
linesearchThe line search algorithm. Default value is using the backtracking with the strong Wolfe condition line search
max_iterationsThe maximum number of iterations for L-BFGS update. Default value is 1000.
deltaDelta for convergence test based on the change of function value. Default value is 0.
pastDistance for delta-based convergence test. Default value is 0.
epsilonEpsilon for convergence test based on the change of gradient. Default value is 1e-5
min_stepThe minimum step of the line search. The default value is 1e-20
max_stepThe maximum step of the line search. The default value is 1e+20
ftolA parameter used in Armijo condition. Default value is 1e-4
wolfeA parameter used in curvature condition. Default value is 0.9
gtolA parameter used in Morethuente linesearch to control the accuracy. Default value is 0.9
xtolThe machine precision for floating-point values. Default value is 1e-16.
orthantwise_cCoeefficient for the L1 norm of variables. This parameter should be set to zero for standard minimization problems. Setting this parameter to a positive value activates Orthant-Wise Limited-memory Quasi-Newton (OWL-QN) method. Default value is 0.
orthantwise_startStart index for computing L1 norm of the variables. This parameter is valid only for OWL-QN method. Default value is 0.
orthantwise_endEnd index for computing L1 norm of the variables. Default value is 1.

Definition at line 58 of file LBFGSMinimizer.cpp.

virtual void set_penalty_type ( Penalty penalty_type)
virtualinherited

Set the type of penalty For example, L2 penalty

Parameters
penalty_typethe type of penalty. If NULL is given, regularization is not enabled.

Definition at line 137 of file FirstOrderMinimizer.h.

virtual void set_penalty_weight ( float64_t  penalty_weight)
virtualinherited

Set the weight of penalty

Parameters
penalty_weightthe weight of penalty, which is positive

Definition at line 126 of file FirstOrderMinimizer.h.

virtual bool supports_batch_update ( ) const
virtual

Does minimizer support batch update?

Returns
whether minimizer supports batch update

Implements FirstOrderMinimizer.

Definition at line 73 of file LBFGSMinimizer.h.

virtual void update_context ( CMinimizerContext context)
protectedvirtualinherited

Update a context object to store mutable variables

Parameters
contexta context object

Reimplemented in FirstOrderStochasticMinimizer, SMDMinimizer, and SMIDASMinimizer.

Definition at line 147 of file FirstOrderMinimizer.h.

virtual void update_gradient ( SGVector< float64_t gradient,
SGVector< float64_t var 
)
protectedvirtualinherited

Add gradient of the penalty wrt target variables to unpenalized gradient For least sqaure with L2 penalty,

\[ L2f(w)=f(w) + L2(w) \]

where \( f(w)=\sum_i{(y_i-w^T x_i)^2}\) is the least sqaure cost function and \(L2(w)=\lambda \frac{w^t w}{2}\) is the L2 penalty

Target variables is \(w\) Unpenalized gradient is \(\frac{\partial f(w) }{\partial w}\) Gradient of the penalty wrt target variables is \(\frac{\partial L2(w) }{\partial w}\)

Parameters
gradientunpenalized gradient wrt its target variable
varthe target variable

Definition at line 190 of file FirstOrderMinimizer.h.

Member Data Documentation

float64_t m_delta
protected

Delta for convergence test based on the change of function value.

Definition at line 174 of file LBFGSMinimizer.h.

float64_t m_epsilon
protected

Epsilon for convergence test based on the change of gradient.

Definition at line 180 of file LBFGSMinimizer.h.

float64_t m_ftol
protected

A parameter used in Armijo condition.

Definition at line 189 of file LBFGSMinimizer.h.

FirstOrderCostFunction* m_fun
protectedinherited

Cost function

Definition at line 205 of file FirstOrderMinimizer.h.

float64_t m_gtol
protected

A parameter used in Morethuente linesearch to control the accuracy.

Definition at line 195 of file LBFGSMinimizer.h.

int m_linesearch
protected

The line search algorithm.

Definition at line 168 of file LBFGSMinimizer.h.

int m_m
protected

The number of corrections to approximate the inverse hessian matrix.

Definition at line 162 of file LBFGSMinimizer.h.

int m_max_iterations
protected

The maximum number of iterations for L-BFGS update.

Definition at line 171 of file LBFGSMinimizer.h.

int m_max_linesearch
protected

The maximum number of trials to do line search for each L-BFGS update.

Definition at line 165 of file LBFGSMinimizer.h.

float64_t m_max_step
protected

The maximum step of the line search.

Definition at line 186 of file LBFGSMinimizer.h.

float64_t m_min_step
protected

The minimum step of the line search.

Definition at line 183 of file LBFGSMinimizer.h.

float64_t m_orthantwise_c
protected

Coeefficient for the L1 norm of variables.

Definition at line 201 of file LBFGSMinimizer.h.

int m_orthantwise_end
protected

End index for computing L1 norm of the variables.

Definition at line 207 of file LBFGSMinimizer.h.

int m_orthantwise_start
protected

Start index for computing L1 norm of the variables.

Definition at line 204 of file LBFGSMinimizer.h.

int m_past
protected

Distance for delta-based convergence test.

Definition at line 177 of file LBFGSMinimizer.h.

Penalty* m_penalty_type
protectedinherited

the type of penalty

Definition at line 208 of file FirstOrderMinimizer.h.

float64_t m_penalty_weight
protectedinherited

the weight of penalty

Definition at line 211 of file FirstOrderMinimizer.h.

SGVector<float64_t> m_target_variable
protected

Target variable

Definition at line 210 of file LBFGSMinimizer.h.

float64_t m_wolfe
protected

A parameter used in curvature condition.

Definition at line 192 of file LBFGSMinimizer.h.

float64_t m_xtol
protected

The machine precision for floating-point values.

Definition at line 198 of file LBFGSMinimizer.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation