The class implements the Stochastic MIrror Descent mAde Sparse (SMIDAS) minimizer.
Reference: Shai Shalev-Shwartz and Ambuj Tewari, Stochastic methods for l1 regularized loss minimization. Proceedings of the 26th International Conference on Machine Learning, pages 929-936, 2009.
在文件 SMIDASMinimizer.h 第 48 行定义.
Public 成员函数 | |
SMIDASMinimizer () | |
SMIDASMinimizer (FirstOrderStochasticCostFunction *fun) | |
virtual | ~SMIDASMinimizer () |
virtual float64_t | minimize () |
virtual void | load_from_context (CMinimizerContext *context) |
virtual void | set_mapping_function (MappingFunction *mapping_fun) |
virtual bool | supports_batch_update () const |
virtual void | set_gradient_updater (DescendUpdater *gradient_updater) |
virtual void | set_number_passes (int32_t num_passes) |
virtual void | set_learning_rate (LearningRate *learning_rate) |
virtual int32_t | get_iteration_counter () |
virtual void | set_cost_function (FirstOrderCostFunction *fun) |
virtual CMinimizerContext * | save_to_context () |
virtual void | set_penalty_weight (float64_t penalty_weight) |
virtual void | set_penalty_type (Penalty *penalty_type) |
Protected 成员函数 | |
virtual void | update_context (CMinimizerContext *context) |
virtual void | init_minimization () |
virtual void | do_proximal_operation (SGVector< float64_t >variable_reference) |
virtual float64_t | get_penalty (SGVector< float64_t > var) |
virtual void | update_gradient (SGVector< float64_t > gradient, SGVector< float64_t > var) |
Protected 属性 | |
SGVector< float64_t > | m_dual_variable |
MappingFunction * | m_mapping_fun |
DescendUpdater * | m_gradient_updater |
int32_t | m_num_passes |
int32_t | m_cur_passes |
int32_t | m_iter_counter |
LearningRate * | m_learning_rate |
FirstOrderCostFunction * | m_fun |
Penalty * | m_penalty_type |
float64_t | m_penalty_weight |
SMIDASMinimizer | ( | ) |
Default constructor
在文件 SMIDASMinimizer.cpp 第 37 行定义.
|
virtual |
Destructor
在文件 SMIDASMinimizer.cpp 第 43 行定义.
|
protectedvirtualinherited |
Do proximal update in place
variable_reference | variable_reference to be updated |
在文件 FirstOrderStochasticMinimizer.h 第 167 行定义.
|
virtualinherited |
How many samples/mini-batch does the minimizer use?
在文件 FirstOrderStochasticMinimizer.h 第 160 行定义.
Get the penalty given target variables For L2 penalty, the target variable is \(w\) and the value of penalty is \(\lambda \frac{w^t w}{2}\), where \(\lambda\) is the weight of penalty
var | the variable used in regularization |
在文件 FirstOrderMinimizer.h 第 164 行定义.
|
protectedvirtual |
|
virtual |
Load the given context object to restores mutable variables Usually it is used in deserialization.
context | a context object |
重载 SMDMinimizer .
在文件 SMIDASMinimizer.h 第 73 行定义.
|
virtual |
Do minimization and get the optimal value
重载 SMDMinimizer .
在文件 SMIDASMinimizer.cpp 第 53 行定义.
|
virtualinherited |
Return a context object which stores mutable variables Usually it is used in serialization.
被 LBFGSMinimizer , 以及 NLOPTMinimizer 重载.
在文件 FirstOrderMinimizer.h 第 103 行定义.
|
virtualinherited |
|
virtualinherited |
Set a gradient updater
gradient_updater | the gradient_updater |
在文件 FirstOrderStochasticMinimizer.h 第 104 行定义.
|
virtualinherited |
Set the learning rate of a minimizer
learning_rate | learn rate or step size |
在文件 FirstOrderStochasticMinimizer.h 第 151 行定义.
|
virtualinherited |
|
virtualinherited |
Set the number of times to go through all data points (samples) For example, num_passes=1 means go through all data points once.
Recall that a stochastic cost function \(f(w)\) can be written as \(\sum_i{ f_i(w) }\), where \(f_i(w)\) is the differentiable function for the i-th sample.
num_passes | the number of times |
在文件 FirstOrderStochasticMinimizer.h 第 125 行定义.
|
virtualinherited |
Set the type of penalty For example, L2 penalty
penalty_type | the type of penalty. If NULL is given, regularization is not enabled. |
在文件 FirstOrderMinimizer.h 第 137 行定义.
|
virtualinherited |
Set the weight of penalty
penalty_weight | the weight of penalty, which is positive |
在文件 FirstOrderMinimizer.h 第 126 行定义.
|
virtualinherited |
Does minimizer support batch update
实现了 FirstOrderMinimizer.
在文件 FirstOrderStochasticMinimizer.h 第 98 行定义.
|
protectedvirtual |
Update a context object to store mutable variables
context | a context object |
重载 SMDMinimizer .
在文件 SMIDASMinimizer.h 第 89 行定义.
|
protectedvirtualinherited |
Add gradient of the penalty wrt target variables to unpenalized gradient For least sqaure with L2 penalty,
\[ L2f(w)=f(w) + L2(w) \]
where \( f(w)=\sum_i{(y_i-w^T x_i)^2}\) is the least sqaure cost function and \(L2(w)=\lambda \frac{w^t w}{2}\) is the L2 penalty
Target variables is \(w\) Unpenalized gradient is \(\frac{\partial f(w) }{\partial w}\) Gradient of the penalty wrt target variables is \(\frac{\partial L2(w) }{\partial w}\)
gradient | unpenalized gradient wrt its target variable |
var | the target variable |
在文件 FirstOrderMinimizer.h 第 190 行定义.
|
protectedinherited |
current iteration to go through data
在文件 FirstOrderStochasticMinimizer.h 第 215 行定义.
dual variable
在文件 SMIDASMinimizer.h 第 104 行定义.
|
protectedinherited |
Cost function
在文件 FirstOrderMinimizer.h 第 205 行定义.
|
protectedinherited |
the gradient update step
在文件 FirstOrderStochasticMinimizer.h 第 200 行定义.
|
protectedinherited |
number of used samples/mini-batches
在文件 FirstOrderStochasticMinimizer.h 第 218 行定义.
|
protectedinherited |
learning_rate object
在文件 FirstOrderStochasticMinimizer.h 第 221 行定义.
|
protectedinherited |
mapping function
在文件 SMDMinimizer.h 第 100 行定义.
|
protectedinherited |
iteration to go through data
在文件 FirstOrderStochasticMinimizer.h 第 212 行定义.
|
protectedinherited |
the type of penalty
在文件 FirstOrderMinimizer.h 第 208 行定义.
|
protectedinherited |
the weight of penalty
在文件 FirstOrderMinimizer.h 第 211 行定义.