This is a base class for descend update with descend based correction.
The class enables descend update with descend-based correction.
Given a target variable, \(w\), and its negative descend direction \(g\), the class will first correct the descend direction, \(g\), and then update \(w\) based on \(g^{corrected}\) (eg, subtracting \(g^{corrected}\))
Note that an example of \(d\) is to simply use the gradient wrt \(w\). An example of using descend based correction can be found at StandardMomentumCorrection
在文件 DescendUpdaterWithCorrection.h 第 52 行定义.
Public 成员函数 | |
virtual | ~DescendUpdaterWithCorrection () |
virtual void | update_variable (SGVector< float64_t > variable_reference, SGVector< float64_t > raw_negative_descend_direction, float64_t learning_rate) |
virtual void | update_context (CMinimizerContext *context) |
virtual void | load_from_context (CMinimizerContext *context) |
virtual void | set_descend_correction (DescendCorrection *correction) |
virtual bool | enables_descend_correction () |
Protected 成员函数 | |
virtual float64_t | get_negative_descend_direction (float64_t variable, float64_t raw_negative_descend_direction, index_t idx, float64_t learning_rate)=0 |
Protected 属性 | |
DescendCorrection * | m_correction |
|
virtual |
在文件 DescendUpdaterWithCorrection.h 第 56 行定义.
|
virtual |
Do we enable descend correction?
在文件 DescendUpdaterWithCorrection.h 第 145 行定义.
|
protectedpure virtual |
Get the negative descend direction given current variable and raw negative descend direction
It will be called by update_variable()
variable | current variable |
raw_negative_descend_direction | current raw negative descend direction |
idx | the index of the variable |
learning_rate | learning rate |
在 AdaDeltaUpdater, RmsPropUpdater, AdaGradUpdater , 以及 GradientDescendUpdater 内被实现.
|
virtual |
Load the given context object to restore mutable variables
This method will be called by FirstOrderMinimizer::load_from_context(CMinimizerContext* context)
context | a context object |
实现了 DescendUpdater.
被 AdaDeltaUpdater, RmsPropUpdater , 以及 AdaGradUpdater 重载.
在文件 DescendUpdaterWithCorrection.h 第 124 行定义.
|
virtual |
Set the type of descend correction
correction | the type of descend correction |
在文件 DescendUpdaterWithCorrection.h 第 135 行定义.
|
virtual |
Update a context object to store mutable variables used in descend update
This method will be called by FirstOrderMinimizer::save_to_context()
context | a context object |
实现了 DescendUpdater.
被 AdaDeltaUpdater, RmsPropUpdater , 以及 AdaGradUpdater 重载.
在文件 DescendUpdaterWithCorrection.h 第 110 行定义.
|
virtual |
Update the target variable based on the given negative descend direction
Note that this method will update the target variable in place. This method will be called by FirstOrderMinimizer::minimize()
variable_reference | a reference of the target variable |
raw_negative_descend_direction | the negative descend direction given the current value |
learning_rate | learning rate |
实现了 DescendUpdater.
被 AdaDeltaUpdater, RmsPropUpdater , 以及 AdaGradUpdater 重载.
在文件 DescendUpdaterWithCorrection.h 第 67 行定义.
|
protected |
descend correction object
在文件 DescendUpdaterWithCorrection.h 第 165 行定义.