Penalty Class Reference

Detailed Description

The base class for penalty/regularization used in minimization.

This is the interface of regularizers used in minimizers. (eg, FirstOrderMinimizer::update_gradient(SGVector<float64_t> gradient, SGVector<float64_t> var) FirstOrderMinimizer::get_penalty(SGVector<float64_t> var) )

Definition at line 46 of file Penalty.h.

Inheritance diagram for Penalty:

Public Member Functions
virtual float64_t	get_penalty (float64_t variable)=0
virtual float64_t	get_penalty_gradient (float64_t variable, float64_t gradient)=0
virtual void	update_context (CMinimizerContext *context)=0
virtual void	load_from_context (CMinimizerContext *context)=0

Member Function Documentation

virtual float64_t get_penalty ( float64_t  variable )

pure virtual

Given the value of a target variable, this method returns the penalty of the variable

For L2 penalty, \(L2(w)\)

\[ L2(w)=\frac{w^t w}{2} \]

Parameters

variable

value of the variable

Returns

penalty of the variable

Implemented in ElasticNetPenalty, L1Penalty, and L2Penalty.

virtual float64_t get_penalty_gradient ( float64_t  variable, float64_t  gradient  )

pure virtual

Returns the gradient of the penalty wrt a target variable Note that the penalized gradient=unpenalized gradient+penalty_gradient

For L2 penalty

\[ \frac{\partial L2(w) }{\partial w}=w \]

Parameters

variable	value of a target variable
gradient	unregularized/unpenalized gradient of the variable

Returns

the gradient of the penalty wrt the variable

Implemented in ElasticNetPenalty, L1Penalty, and L2Penalty.

virtual void load_from_context ( CMinimizerContext *  context )

pure virtual

Load the given context object to restore mutable variables

Parameters

context

a context object

Implemented in ElasticNetPenalty, L1Penalty, L1PenaltyForTG, and L2Penalty.

virtual void update_context ( CMinimizerContext *  context )

pure virtual

Update a context object to store mutable variables used in learning rate

Parameters

context

a context object

Implemented in ElasticNetPenalty, L1Penalty, L1PenaltyForTG, and L2Penalty.

---

The documentation for this class was generated from the following file:

• Penalty.h