SHOGUN  3.2.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Classes | Typedefs | Enumerations | Functions
libLBFGS API

Classes

struct  lbfgs_parameter_t

Typedefs

typedef float64_t(* lbfgs_evaluate_t )(void *instance, const float64_t *x, float64_t *g, const int n, const float64_t step)
typedef int(* lbfgs_progress_t )(void *instance, const float64_t *x, const float64_t *g, const float64_t fx, const float64_t xnorm, const float64_t gnorm, const float64_t step, int n, int k, int ls)
typedef float64_t(* lbfgs_adjust_step_t )(void *instance, const float64_t *x, const float64_t *d, const int n, const float64_t step)

Enumerations

enum  {
  LBFGS_SUCCESS = 0, LBFGS_CONVERGENCE = 0, LBFGS_STOP, LBFGS_ALREADY_MINIMIZED,
  LBFGSERR_UNKNOWNERROR = -1024, LBFGSERR_LOGICERROR, LBFGSERR_OUTOFMEMORY, LBFGSERR_CANCELED,
  LBFGSERR_INVALID_N, LBFGSERR_INVALID_N_SSE, LBFGSERR_INVALID_X_SSE, LBFGSERR_INVALID_EPSILON,
  LBFGSERR_INVALID_TESTPERIOD, LBFGSERR_INVALID_DELTA, LBFGSERR_INVALID_LINESEARCH, LBFGSERR_INVALID_MINSTEP,
  LBFGSERR_INVALID_MAXSTEP, LBFGSERR_INVALID_FTOL, LBFGSERR_INVALID_WOLFE, LBFGSERR_INVALID_GTOL,
  LBFGSERR_INVALID_XTOL, LBFGSERR_INVALID_MAXLINESEARCH, LBFGSERR_INVALID_ORTHANTWISE, LBFGSERR_INVALID_ORTHANTWISE_START,
  LBFGSERR_INVALID_ORTHANTWISE_END, LBFGSERR_OUTOFINTERVAL, LBFGSERR_INCORRECT_TMINMAX, LBFGSERR_ROUNDING_ERROR,
  LBFGSERR_MINIMUMSTEP, LBFGSERR_MAXIMUMSTEP, LBFGSERR_MAXIMUMLINESEARCH, LBFGSERR_MAXIMUMITERATION,
  LBFGSERR_WIDTHTOOSMALL, LBFGSERR_INVALIDPARAMETERS, LBFGSERR_INCREASEGRADIENT, LBFGSERR_INVALID_VALUE
}
enum  {
  LBFGS_LINESEARCH_DEFAULT = 0, LBFGS_LINESEARCH_MORETHUENTE = 0, LBFGS_LINESEARCH_BACKTRACKING_ARMIJO = 1, LBFGS_LINESEARCH_BACKTRACKING = 2,
  LBFGS_LINESEARCH_BACKTRACKING_WOLFE = 2, LBFGS_LINESEARCH_BACKTRACKING_STRONG_WOLFE = 3
}

Functions

int lbfgs (int n, float64_t *x, float64_t *ptr_fx, lbfgs_evaluate_t proc_evaluate, lbfgs_progress_t proc_progress, void *instance, lbfgs_parameter_t *param, lbfgs_adjust_step_t proc_adjust_step=NULL)
void lbfgs_parameter_init (lbfgs_parameter_t *param)

Detailed Description

The libLBFGS API.

Typedef Documentation

typedef float64_t(* lbfgs_adjust_step_t)(void *instance, const float64_t *x, const float64_t *d, const int n, const float64_t step)

Callback interface to adjust step size based on constraints.

If the function pointer is not NULL, the lbfgs() function calls this function to adjust its step size. A client program can implement this function to adjust the step size used in lbfgs update based on user-defined constraints. Note that the update is x_new = x + step * d, where step is positive.

Parameters
instanceThe user data sent for lbfgs() function by the client.
xThe current values of variables.
dThe direction vector of variables.
nThe number of variables.
stepThe current step of the line search routine.
Return values
float64_tThe value of adjusted step size

Definition at line 413 of file lbfgs.h.

typedef float64_t(* lbfgs_evaluate_t)(void *instance, const float64_t *x, float64_t *g, const int n, const float64_t step)

Callback interface to provide objective function and gradient evaluations.

The lbfgs() function call this function to obtain the values of objective function and its gradients when needed. A client program must implement this function to evaluate the values of the objective function and its gradients, given current values of variables.

Parameters
instanceThe user data sent for lbfgs() function by the client.
xThe current values of variables.
gThe gradient vector. The callback function must compute the gradient values for the current variables.
nThe number of variables.
stepThe current step of the line search routine.
Return values
float64_tThe value of the objective function for the current variables.

Definition at line 355 of file lbfgs.h.

typedef int(* lbfgs_progress_t)(void *instance, const float64_t *x, const float64_t *g, const float64_t fx, const float64_t xnorm, const float64_t gnorm, const float64_t step, int n, int k, int ls)

Callback interface to receive the progress of the optimization process.

The lbfgs() function call this function for each iteration. Implementing this function, a client program can store or display the current progress of the optimization process.

Parameters
instanceThe user data sent for lbfgs() function by the client.
xThe current values of variables.
gThe current gradient values of variables.
fxThe current value of the objective function.
xnormThe Euclidean norm of the variables.
gnormThe Euclidean norm of the gradients.
stepThe line-search step used for this iteration.
nThe number of variables.
kThe iteration count.
lsThe number of evaluations called for this iteration.
Return values
intZero to continue the optimization process. Returning a non-zero value will cancel the optimization process.

Definition at line 383 of file lbfgs.h.

Enumeration Type Documentation

anonymous enum

Return values of lbfgs().

Roughly speaking, a negative value indicates an error.

Enumerator:
LBFGS_SUCCESS 

L-BFGS reaches convergence.

LBFGS_CONVERGENCE 
LBFGS_STOP 
LBFGS_ALREADY_MINIMIZED 

The initial variables already minimize the objective function.

LBFGSERR_UNKNOWNERROR 

Unknown error.

LBFGSERR_LOGICERROR 

Logic error.

LBFGSERR_OUTOFMEMORY 

Insufficient memory.

LBFGSERR_CANCELED 

The minimization process has been canceled.

LBFGSERR_INVALID_N 

Invalid number of variables specified.

LBFGSERR_INVALID_N_SSE 

Invalid number of variables (for SSE) specified.

LBFGSERR_INVALID_X_SSE 

The array x must be aligned to 16 (for SSE).

LBFGSERR_INVALID_EPSILON 

Invalid parameter lbfgs_parameter_t::epsilon specified.

LBFGSERR_INVALID_TESTPERIOD 

Invalid parameter lbfgs_parameter_t::past specified.

LBFGSERR_INVALID_DELTA 

Invalid parameter lbfgs_parameter_t::delta specified.

LBFGSERR_INVALID_LINESEARCH 

Invalid parameter lbfgs_parameter_t::linesearch specified.

LBFGSERR_INVALID_MINSTEP 

Invalid parameter lbfgs_parameter_t::max_step specified.

LBFGSERR_INVALID_MAXSTEP 

Invalid parameter lbfgs_parameter_t::max_step specified.

LBFGSERR_INVALID_FTOL 

Invalid parameter lbfgs_parameter_t::ftol specified.

LBFGSERR_INVALID_WOLFE 

Invalid parameter lbfgs_parameter_t::wolfe specified.

LBFGSERR_INVALID_GTOL 

Invalid parameter lbfgs_parameter_t::gtol specified.

LBFGSERR_INVALID_XTOL 

Invalid parameter lbfgs_parameter_t::xtol specified.

LBFGSERR_INVALID_MAXLINESEARCH 

Invalid parameter lbfgs_parameter_t::max_linesearch specified.

LBFGSERR_INVALID_ORTHANTWISE 

Invalid parameter lbfgs_parameter_t::orthantwise_c specified.

LBFGSERR_INVALID_ORTHANTWISE_START 

Invalid parameter lbfgs_parameter_t::orthantwise_start specified.

LBFGSERR_INVALID_ORTHANTWISE_END 

Invalid parameter lbfgs_parameter_t::orthantwise_end specified.

LBFGSERR_OUTOFINTERVAL 

The line-search step went out of the interval of uncertainty.

LBFGSERR_INCORRECT_TMINMAX 

A logic error occurred; alternatively, the interval of uncertainty became too small.

LBFGSERR_ROUNDING_ERROR 

A rounding error occurred; alternatively, no line-search step satisfies the sufficient decrease and curvature conditions.

LBFGSERR_MINIMUMSTEP 

The line-search step became smaller than lbfgs_parameter_t::min_step.

LBFGSERR_MAXIMUMSTEP 

The line-search step became larger than lbfgs_parameter_t::max_step.

LBFGSERR_MAXIMUMLINESEARCH 

The line-search routine reaches the maximum number of evaluations.

LBFGSERR_MAXIMUMITERATION 

The algorithm routine reaches the maximum number of iterations.

LBFGSERR_WIDTHTOOSMALL 

Relative width of the interval of uncertainty is at most lbfgs_parameter_t::xtol.

LBFGSERR_INVALIDPARAMETERS 

A logic error (negative line-search step) occurred.

LBFGSERR_INCREASEGRADIENT 

The current search direction increases the objective function value.

LBFGSERR_INVALID_VALUE 

Invalid value of objective function

Definition at line 49 of file lbfgs.h.

anonymous enum

Line search algorithms.

Enumerator:
LBFGS_LINESEARCH_DEFAULT 

The default algorithm (MoreThuente method).

LBFGS_LINESEARCH_MORETHUENTE 

MoreThuente method proposd by More and Thuente.

LBFGS_LINESEARCH_BACKTRACKING_ARMIJO 

Backtracking method with the Armijo condition. The backtracking method finds the step length such that it satisfies the sufficient decrease (Armijo) condition,

where x is the current point, d is the current search direction, and a is the step length.

LBFGS_LINESEARCH_BACKTRACKING 

The backtracking method with the defualt (regular Wolfe) condition.

LBFGS_LINESEARCH_BACKTRACKING_WOLFE 

Backtracking method with regular Wolfe condition. The backtracking method finds the step length such that it satisfies both the Armijo condition (LBFGS_LINESEARCH_BACKTRACKING_ARMIJO) and the curvature condition,

where x is the current point, d is the current search direction, and a is the step length.

LBFGS_LINESEARCH_BACKTRACKING_STRONG_WOLFE 

Backtracking method with strong Wolfe condition. The backtracking method finds the step length such that it satisfies both the Armijo condition (LBFGS_LINESEARCH_BACKTRACKING_ARMIJO) and the following condition,

where x is the current point, d is the current search direction, and a is the step length.

Definition at line 129 of file lbfgs.h.

Function Documentation

int shogun::lbfgs ( int  n,
float64_t x,
float64_t ptr_fx,
lbfgs_evaluate_t  proc_evaluate,
lbfgs_progress_t  proc_progress,
void *  instance,
lbfgs_parameter_t *  param,
lbfgs_adjust_step_t  proc_adjust_step = NULL 
)

Start a L-BFGS optimization.

Parameters
nThe number of variables.
xThe array of variables. A client program can set default values for the optimization and receive the optimization result through this array. This array must be allocated by lbfgs_malloc function for libLBFGS built with SSE/SSE2 optimization routine enabled. The library built without SSE/SSE2 optimization does not have such a requirement.
ptr_fxThe pointer to the variable that receives the final value of the objective function for the variables. This argument can be set to NULL if the final value of the objective function is unnecessary.
proc_evaluateThe callback function to provide function and gradient evaluations given a current values of variables. A client program must implement a callback function compatible with lbfgs_evaluate_t and pass the pointer to the callback function.
proc_progressThe callback function to receive the progress (the number of iterations, the current value of the objective function) of the minimization process. This argument can be set to NULL if a progress report is unnecessary.
instanceA user data for the client program. The callback functions will receive the value of this argument.
paramThe pointer to a structure representing parameters for L-BFGS optimization. A client program can set this parameter to NULL to use the default parameters. Call lbfgs_parameter_init() function to fill a structure with the default values.
proc_adjust_stepThe callback function to adjust step size based on constraints. This argument can be set to NULL if there is not constraint.
Return values
intThe status code. This function returns zero if the minimization process terminates without an error. A non-zero value indicates an error.
void lbfgs_parameter_init ( lbfgs_parameter_t *  param)

Initialize L-BFGS parameters to the default values.

Call this function to fill a parameter structure with the default values and overwrite parameter values if necessary.

Parameters
paramThe pointer to the parameter structure.

Definition at line 203 of file lbfgs.cpp.


SHOGUN Machine Learning Toolbox - Documentation