FirstOrderMinimizer Class Reference

Detailed Description

The first order minimizer base class.

This class gives the interface of a first-order gradient-based unconstrained minimizer

This kind of minimizers will find optimal target variables based on gradient information wrt target variables. For example, the gradient descend method is a minimizer.

A minimizer requires the following objects as input: a supported cost function object (eg, FirstOrderCostFunction ) a penalty object if regularization is enabled (eg, Penalty ) a context object to restore mutable variables if deserialization is actived (eg, CMinimizerContext )

Definition at line 54 of file FirstOrderMinimizer.h.

Inheritance diagram for FirstOrderMinimizer:

Public Member Functions
	FirstOrderMinimizer ()
	FirstOrderMinimizer (FirstOrderCostFunction *fun)
virtual	~FirstOrderMinimizer ()
virtual float64_t	minimize ()=0
virtual bool	supports_batch_update () const =0
virtual void	set_cost_function (FirstOrderCostFunction *fun)
virtual CMinimizerContext *	save_to_context ()
virtual void	load_from_context (CMinimizerContext *context)
virtual void	set_penalty_weight (float64_t penalty_weight)
virtual void	set_penalty_type (Penalty *penalty_type)

Protected Member Functions
virtual void	update_context (CMinimizerContext *context)
virtual float64_t	get_penalty (SGVector< float64_t > var)
virtual void	update_gradient (SGVector< float64_t > gradient, SGVector< float64_t > var)

Protected Attributes
FirstOrderCostFunction *	m_fun
Penalty *	m_penalty_type
float64_t	m_penalty_weight

Constructor & Destructor Documentation

FirstOrderMinimizer

(

)

Default constructor

Definition at line 58 of file FirstOrderMinimizer.h.

FirstOrderMinimizer

(

FirstOrderCostFunction *

fun

)

Constructor

Parameters

fun	cost function (user have to manully delete the pointer)

Definition at line 66 of file FirstOrderMinimizer.h.

virtual ~FirstOrderMinimizer ( )

virtual

Destructor

Definition at line 73 of file FirstOrderMinimizer.h.

Member Function Documentation

virtual float64_t get_penalty ( SGVector< float64_t >  var )

protectedvirtual

Get the penalty given target variables For L2 penalty, the target variable is \(w\) and the value of penalty is \(\lambda \frac{w^t w}{2}\), where \(\lambda\) is the weight of penalty

Parameters

var	the variable used in regularization

Definition at line 164 of file FirstOrderMinimizer.h.

virtual void load_from_context ( CMinimizerContext *  context )

virtual

Load the given context object to restores mutable variables Usually it is used in deserialization.

Parameters

context

a context object

Reimplemented in LBFGSMinimizer, FirstOrderStochasticMinimizer, NLOPTMinimizer, SMDMinimizer, and SMIDASMinimizer.

Definition at line 115 of file FirstOrderMinimizer.h.

virtual float64_t minimize ( )

pure virtual

Do minimization and get the optimal value

Returns

optimal value

Implemented in FirstOrderStochasticMinimizer, LBFGSMinimizer, SMIDASMinimizer, NLOPTMinimizer, SVRGMinimizer, SGDMinimizer, and SMDMinimizer.

virtual CMinimizerContext* save_to_context ( )

virtual

Return a context object which stores mutable variables Usually it is used in serialization.

Returns

a context object

Reimplemented in LBFGSMinimizer, and NLOPTMinimizer.

Definition at line 103 of file FirstOrderMinimizer.h.

virtual void set_cost_function ( FirstOrderCostFunction *  fun )

virtual

Set cost function used in the minimizer

Parameters

fun	the cost function

Definition at line 92 of file FirstOrderMinimizer.h.

virtual void set_penalty_type ( Penalty *  penalty_type )

virtual

Set the type of penalty For example, L2 penalty

Parameters

penalty_type

the type of penalty. If NULL is given, regularization is not enabled.

Definition at line 137 of file FirstOrderMinimizer.h.

virtual void set_penalty_weight ( float64_t  penalty_weight )

virtual

Set the weight of penalty

Parameters

penalty_weight

the weight of penalty, which is positive

Definition at line 126 of file FirstOrderMinimizer.h.

virtual bool supports_batch_update ( ) const

pure virtual

Does minimizer support batch update?

Returns

whether minimizer supports batch update

Implemented in FirstOrderStochasticMinimizer, LBFGSMinimizer, and NLOPTMinimizer.

virtual void update_context ( CMinimizerContext *  context )

protectedvirtual

Update a context object to store mutable variables

Parameters

context

a context object

Reimplemented in FirstOrderStochasticMinimizer, SMDMinimizer, and SMIDASMinimizer.

Definition at line 147 of file FirstOrderMinimizer.h.

virtual void update_gradient ( SGVector< float64_t >  gradient, SGVector< float64_t >  var  )

protectedvirtual

Add gradient of the penalty wrt target variables to unpenalized gradient For least sqaure with L2 penalty,

\[ L2f(w)=f(w) + L2(w) \]

where \( f(w)=\sum_i{(y_i-w^T x_i)^2}\) is the least sqaure cost function and \(L2(w)=\lambda \frac{w^t w}{2}\) is the L2 penalty

Target variables is \(w\) Unpenalized gradient is \(\frac{\partial f(w) }{\partial w}\) Gradient of the penalty wrt target variables is \(\frac{\partial L2(w) }{\partial w}\)

Parameters

gradient	unpenalized gradient wrt its target variable
var	the target variable

Definition at line 190 of file FirstOrderMinimizer.h.

Member Data Documentation

FirstOrderCostFunction* m_fun

protected

Cost function

Definition at line 205 of file FirstOrderMinimizer.h.

Penalty* m_penalty_type

protected

the type of penalty

Definition at line 208 of file FirstOrderMinimizer.h.

float64_t m_penalty_weight

protected

the weight of penalty

Definition at line 211 of file FirstOrderMinimizer.h.

---

The documentation for this class was generated from the following file:

• FirstOrderMinimizer.h