L1Penalty Class Reference

Detailed Description

The is the base class for L1 penalty/regularization within the FirstOrderMinimizer framework.

For L1 penalty, \(L1(w)\)

\[ L1(w)=\| w \|_1 = \sum_i \| w_i \| \]

This class implements the soft-threshold method. Reference: Proximal gradient method www.eecs.berkeley.edu/~elghaoui/Teaching/EE227A/lecture18.pdf

Definition at line 52 of file L1Penalty.h.

Inheritance diagram for L1Penalty:

Public Member Functions
	L1Penalty ()
virtual	~L1Penalty ()
virtual float64_t	get_penalty (float64_t variable)
virtual float64_t	get_penalty_gradient (float64_t variable, float64_t gradient_of_variable)
virtual void	set_rounding_epsilon (float64_t epsilon)
virtual void	update_variable_for_proximity (SGVector< float64_t > variable, float64_t proximal_weight)
virtual void	update_context (CMinimizerContext *context)
virtual void	load_from_context (CMinimizerContext *context)
virtual float64_t	get_sparse_variable (float64_t variable, float64_t penalty_weight)

Protected Attributes
float64_t	m_rounding_epsilon

Constructor & Destructor Documentation

L1Penalty

(

)

Definition at line 55 of file L1Penalty.h.

virtual ~L1Penalty ( )

virtual

Definition at line 58 of file L1Penalty.h.

Member Function Documentation

virtual float64_t get_penalty ( float64_t  variable )

virtual

Given the value of a target variable, this method returns the penalty of the variable

Parameters

variable

value of the variable

Returns

penalty of the variable

Implements Penalty.

Definition at line 66 of file L1Penalty.h.

virtual float64_t get_penalty_gradient ( float64_t  variable, float64_t  gradient_of_variable  )

virtual

Return the gradient of the penalty wrt a target variable

Note that for L1 penalty we do not compute the gradient/sub-gradient in our implementation. Instead, we do a proximal projection.

Parameters

variable	value of a target variable
gradient_of_variable	unregularized/unpenalized gradient of the variable

Returns

the gradient of the penalty wrt the variable

Implements Penalty.

Definition at line 79 of file L1Penalty.h.

virtual float64_t get_sparse_variable ( float64_t  variable, float64_t  penalty_weight  )

virtual

Get the sparse variable

Parameters

variable	the raw variable
penalty_weight	weight of the penalty

Returns

sparse value of the variable

Implements SparsePenalty.

Definition at line 128 of file L1Penalty.h.

virtual void load_from_context ( CMinimizerContext *  context )

virtual

Load the given context object to restore mutable variables

Parameters

context

a context object

Implements Penalty.

Reimplemented in L1PenaltyForTG.

Definition at line 118 of file L1Penalty.h.

virtual void set_rounding_epsilon ( float64_t  epsilon )

virtual

Set the rounding epsilon

Parameters

epsilon

rounding epsilon

Definition at line 87 of file L1Penalty.h.

virtual void update_context ( CMinimizerContext *  context )

virtual

Update a context object to store mutable variables used in learning rate

Parameters

context

a context object

Implements Penalty.

Reimplemented in L1PenaltyForTG.

Definition at line 109 of file L1Penalty.h.

virtual void update_variable_for_proximity ( SGVector< float64_t >  variable, float64_t  proximal_weight  )

virtual

Do proximal projection/operation in place

Parameters

variable	the raw variable
proximal_weight	weight of the penalty

Implements ProximalPenalty.

Reimplemented in L1PenaltyForTG.

Definition at line 97 of file L1Penalty.h.

Member Data Documentation

float64_t m_rounding_epsilon

protected

rounding epsilon

Definition at line 148 of file L1Penalty.h.

---

The documentation for this class was generated from the following file:

• L1Penalty.h