GradientDescendUpdater Class Reference

Detailed Description

The class implements the gradient descend method.

Given a target variable, \(w\), and its gradient, \(d\), without gradient correction (eg, momentum correction), this class performs the following update.

\[ w^{new} = w - \lambda d \]

, where \(\lambda\) is a learning rate.

Definition at line 48 of file GradientDescendUpdater.h.

Inheritance diagram for GradientDescendUpdater:

Public Member Functions
	GradientDescendUpdater ()
virtual	~GradientDescendUpdater ()
virtual void	update_variable (SGVector< float64_t > variable_reference, SGVector< float64_t > raw_negative_descend_direction, float64_t learning_rate)
virtual void	update_context (CMinimizerContext *context)
virtual void	load_from_context (CMinimizerContext *context)
virtual void	set_descend_correction (DescendCorrection *correction)
virtual bool	enables_descend_correction ()

Protected Member Functions
virtual float64_t	get_negative_descend_direction (float64_t variable, float64_t gradient, index_t idx, float64_t learning_rate)

Protected Attributes
DescendCorrection *	m_correction

Constructor & Destructor Documentation

GradientDescendUpdater

(

)

Definition at line 36 of file GradientDescendUpdater.cpp.

~GradientDescendUpdater ( )

virtual

Definition at line 42 of file GradientDescendUpdater.cpp.

Member Function Documentation

virtual bool enables_descend_correction ( )

virtualinherited

Do we enable descend correction?

Returns

whether we enable descend correction

Definition at line 145 of file DescendUpdaterWithCorrection.h.

float64_t get_negative_descend_direction ( float64_t  variable, float64_t  gradient, index_t  idx, float64_t  learning_rate  )

protectedvirtual

Get the negative descend direction given current variable and gradient

It will be called at update_variable()

Parameters

variable	current variable
gradient	current gradient
idx	the index of the variable
learning_rate	learning rate

Returns

negative descend direction (that is, the given gradient in the class)

Implements DescendUpdaterWithCorrection.

Definition at line 50 of file GradientDescendUpdater.cpp.

virtual void load_from_context ( CMinimizerContext *  context )

virtualinherited

Load the given context object to restore mutable variables

This method will be called by FirstOrderMinimizer::load_from_context(CMinimizerContext* context)

Parameters

context

a context object

Implements DescendUpdater.

Reimplemented in AdaDeltaUpdater, RmsPropUpdater, and AdaGradUpdater.

Definition at line 124 of file DescendUpdaterWithCorrection.h.

virtual void set_descend_correction ( DescendCorrection *  correction )

virtualinherited

Set the type of descend correction

Parameters

correction

the type of descend correction

Definition at line 135 of file DescendUpdaterWithCorrection.h.

virtual void update_context ( CMinimizerContext *  context )

virtualinherited

Update a context object to store mutable variables used in descend update

This method will be called by FirstOrderMinimizer::save_to_context()

Parameters

context

a context object

Implements DescendUpdater.

Reimplemented in AdaDeltaUpdater, RmsPropUpdater, and AdaGradUpdater.

Definition at line 110 of file DescendUpdaterWithCorrection.h.

virtual void update_variable ( SGVector< float64_t >  variable_reference, SGVector< float64_t >  raw_negative_descend_direction, float64_t  learning_rate  )

virtualinherited

Update the target variable based on the given negative descend direction

Note that this method will update the target variable in place. This method will be called by FirstOrderMinimizer::minimize()

Parameters

variable_reference	a reference of the target variable
raw_negative_descend_direction	the negative descend direction given the current value
learning_rate	learning rate

Implements DescendUpdater.

Reimplemented in AdaDeltaUpdater, RmsPropUpdater, and AdaGradUpdater.

Definition at line 67 of file DescendUpdaterWithCorrection.h.

Member Data Documentation

DescendCorrection* m_correction

protectedinherited

descend correction object

Definition at line 165 of file DescendUpdaterWithCorrection.h.

---

The documentation for this class was generated from the following files:

• GradientDescendUpdater.h
• GradientDescendUpdater.cpp