forum.alglib.net

To run ALGLIB LBFGS optimisation solver, I noticed that we have to provide both the analytical function and its analytical gradient. But sometimes the analytical form of the gradient cannot always be calculated for some complex functions. To avoid such problem, we have to estimate the gradient through a four point formula computed through finite difference method.

I think that it would be nice to have the possibility to ask LBFGS solver to estimate directly the gradiant in the core code when its analytical form is not provided. This could be applied to all ALGLIB optimisation solvers.

What do you think?

Kind regards,
Antoine

I've thought about that, but there are exist difficulties associated with a) algorithm tolerance to errors in the gradient, and b) selection of proper differentiation step. For example, Levenberg-Marquardt allows moderate errors in the gradient, but CG requires precise gradient.

I think that L-BFGS is somewhere between LM and CG, and it is worth a try. Maybe even in the future ALGLIB 3.3 - I plan to improve numerical optimization significantly, it may be worth addition.