forum.alglib.net

Hello everyone,
I would like to know if there is a possibility that I can know what step size the LBFGS algorithm is deciding on taking ? I know I can set the maximum step size but I will like to know specifically what is the step size its taking and if it also possible to get the Hessian matrix as output ?


On an unrelated topic, I was wondering if I can somehow trick teh algorithm to perform like steepest descent ? If someone knows any other good library for steepest descent please suggest

Thanks
Vikash

1. you may call minlbfgssetxrep(...,true) to turn on reports about algorithm progress, and then you will be able to compare current point with last one reported by algorithm (last point for which algorithm accepted step).

2. it is impossible to return Hessian matrix, algorithm does not evaluate it explicitly.

3. in order to get steepest descent, you may use MinCG subpackage, and manually rewrite it a bit. Find lines below "// Choose BetaK" comment, and rewrite them by "betak = 0.0". It will force CG algorithm to behave like steepest descent.

Or, alternatively, you may add your own value for State.CGType field (you will understand what I mean if you look at this piece of code) - say, CGtype=-1, which will denote "steepest descent" - and add "if( state.cgtype==-1 ) betak = 0;". So you will be able to have both steepest descent and unmodified CG algorithm.

Hello,
I understand that the library probably is not computing the Hessian explicitly. However, it will be nice to know what kind of estimation it is using, for example if it is using the Sherman-Morrison-Woodbery to estimate the Hessian.

In this case I opened the optimization.cpp I found these lines

I added

below this.

However when I run the code it gives me malloc errors
*** glibc detected *** "path": malloc(): memory corruption: 0x0000000001b681a0 ***


Thanks a lot
Vikash

Hmmm.... strange. I have to check it myself. i will get back to you tomorrow.

Hi Sergey,
No its fine... I debugged, it was because I was writing
mincgcreate(1, x, state) Now it doesnt give that error anymore..
I am just concerned if setting betak =0 really makes it the classical steepest descent ?

Thanks
Vikash

Hello! Thanks for info on the reasons behind failure.

Regarding steepest descent... Yes, it will be classical steepest descent (line search across gradient vector), assuming that: a) betak set to zero, b) preconditioning is turned off (you called minlbfgssetprecdiag() with unit vector as preconditioner).

Hi,
I am using minlbfgssetxrep(state ,1) . Still I dont get the step size. Can you tell me where it is stored as in state.xxx which gives the step size ?

Thanks
Vikash

What compiler you used?

_________________
zoha