|
SHOGUN
4.1.0
|
This implements the plain momentum correction.
Given a target variable, \(w\), and a current descend direction, \(d\), the momentum method performs the following update:
\begin{eqnarray*} v^{new} &=& \mu v - d \\ w^{new} &=& w + v^{new} \end{eqnarray*}
where \(\mu\) is a momentum, \(v\) is a previous descend direction, \(d\) is a current descend direction (eg, \(d=\lambda g\), where \(\lambda\) is a learn rate, \(g\) is gradient), and \(v^{new}\) is a corrected descend direction.
The get_corrected_descend_direction() method will do
\[ v^{new} = \mu v - d \]
and return \(v^{new}\)
A good introduction to the momentum update can be found at http://cs231n.github.io/neural-networks-3/#sgd
If you read the introduction at http://cs231n.github.io/neural-networks-3/#sgd , you may know that \(v\) is also called velocity.
在文件 StandardMomentumCorrection.h 第 62 行定义.
Public 成员函数 | |
| StandardMomentumCorrection () | |
| virtual | ~StandardMomentumCorrection () |
| virtual DescendPair | get_corrected_descend_direction (float64_t negative_descend_direction, index_t idx) |
| virtual bool | is_initialized () |
| virtual void | initialize_previous_direction (index_t len) |
| virtual void | update_context (CMinimizerContext *context) |
| virtual void | load_from_context (CMinimizerContext *context) |
| virtual float64_t | get_previous_descend_direction (index_t idx) |
| virtual float64_t | get_length_previous_descend_direction () |
| virtual void | set_correction_weight (float64_t weight) |
Protected 属性 | |
| SGVector< float64_t > | m_previous_descend_direction |
| float64_t | m_weight |
在文件 StandardMomentumCorrection.h 第 66 行定义.
|
virtual |
在文件 StandardMomentumCorrection.h 第 73 行定义.
|
virtual |
Get corrected descend direction
| negative_descend_direction | the negative descend direction |
| idx | the index of the direction |
实现了 MomentumCorrection.
在文件 StandardMomentumCorrection.h 第 82 行定义.
|
virtualinherited |
Get the length of the previous descend direction (velocity)
在文件 MomentumCorrection.h 第 143 行定义.
Get the previous descend direction (velocity) given the index
| idx | index of the previous descend direction |
在文件 MomentumCorrection.h 第 132 行定义.
|
virtualinherited |
Initialize m_previous_descend_direction?
被 AdaptMomentumCorrection 重载.
在文件 MomentumCorrection.h 第 73 行定义.
|
virtualinherited |
Is the m_previous_descend_direction initialized?
被 AdaptMomentumCorrection 重载.
在文件 MomentumCorrection.h 第 64 行定义.
|
virtualinherited |
Load the given context object to restore mutable variables
This method will be called by DescendUpdaterWithCorrection::load_from_context(CMinimizerContext* context)
| context | a context object |
实现了 DescendCorrection.
被 AdaptMomentumCorrection 重载.
在文件 MomentumCorrection.h 第 116 行定义.
|
virtualinherited |
Set the weight used in descend correction
param weight the weight
被 AdaptMomentumCorrection 重载.
在文件 DescendCorrection.h 第 74 行定义.
|
virtualinherited |
Update a context object to store mutable variables used in descend update
This method will be called by DescendUpdaterWithCorrection::update_context()
| context | a context object |
实现了 DescendCorrection.
被 AdaptMomentumCorrection 重载.
在文件 MomentumCorrection.h 第 98 行定义.
used in momentum methods
在文件 MomentumCorrection.h 第 149 行定义.
|
protectedinherited |
weight of correction
在文件 DescendCorrection.h 第 110 行定义.
