ProximalPenalty Class Reference

Detailed Description

The base class for sparse penalty/regularization used in minimization.

This is the interface of regularizers used in minimizers. (eg, FirstOrderMinimizer::update_gradient(SGVector<float64_t> gradient, SGVector<float64_t> var) FirstOrderMinimizer::get_penalty(SGVector<float64_t> var) )

Reference: Proximal gradient method www.eecs.berkeley.edu/~elghaoui/Teaching/EE227A/lecture18.pdf

Definition at line 50 of file ProximalPenalty.h.

Inheritance diagram for ProximalPenalty:

Public Member Functions
virtual void	update_variable_for_proximity (SGVector< float64_t > variable, float64_t proximal_weight)=0
virtual float64_t	get_penalty (float64_t variable)=0
virtual float64_t	get_penalty_gradient (float64_t variable, float64_t gradient)=0
virtual void	update_context (CMinimizerContext *context)=0
virtual void	load_from_context (CMinimizerContext *context)=0

Member Function Documentation

virtual float64_t get_penalty ( float64_t  variable )

pure virtualinherited

Given the value of a target variable, this method returns the penalty of the variable

For L2 penalty, \(L2(w)\)

\[ L2(w)=\frac{w^t w}{2} \]

Parameters

variable

value of the variable

Returns

penalty of the variable

Implemented in ElasticNetPenalty, L1Penalty, and L2Penalty.

virtual float64_t get_penalty_gradient ( float64_t  variable, float64_t  gradient  )

pure virtualinherited

Returns the gradient of the penalty wrt a target variable Note that the penalized gradient=unpenalized gradient+penalty_gradient

For L2 penalty

\[ \frac{\partial L2(w) }{\partial w}=w \]

Parameters

variable	value of a target variable
gradient	unregularized/unpenalized gradient of the variable

Returns

the gradient of the penalty wrt the variable

Implemented in ElasticNetPenalty, L1Penalty, and L2Penalty.

virtual void load_from_context ( CMinimizerContext *  context )

pure virtualinherited

Load the given context object to restore mutable variables

Parameters

context

a context object

Implemented in ElasticNetPenalty, L1Penalty, L1PenaltyForTG, and L2Penalty.

virtual void update_context ( CMinimizerContext *  context )

pure virtualinherited

Update a context object to store mutable variables used in learning rate

Parameters

context

a context object

Implemented in ElasticNetPenalty, L1Penalty, L1PenaltyForTG, and L2Penalty.

virtual void update_variable_for_proximity ( SGVector< float64_t >  variable, float64_t  proximal_weight  )

pure virtual

Do proximal projection/operation in place

Parameters

variable	the raw variable
proximal_weight	weight of the penalty

Implemented in ElasticNetPenalty, L1Penalty, and L1PenaltyForTG.

---

The documentation for this class was generated from the following file:

• ProximalPenalty.h