forum.alglib.net
http://forum.alglib.net/

How do I know what step size the LBFGS algorithm is taking
http://forum.alglib.net/viewtopic.php?f=2&t=1041		 Page 1 of 1
Author:  vigupta [ Tue Nov 05, 2013 12:11 am ]
Post subject:  How do I know what step size the LBFGS algorithm is taking
Hello everyone,
I would like to know if there is a possibility that I can know what step size the LBFGS algorithm is deciding on taking ? I know I can set the maximum step size but I will like to know specifically what is the step size its taking and if it also possible to get the Hessian matrix as output ?


On an unrelated topic, I was wondering if I can somehow trick teh algorithm to perform like steepest descent ? If someone knows any other good library for steepest descent please suggest

Thanks
Vikash
Author:  Sergey.Bochkanov [ Tue Nov 05, 2013 1:27 pm ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
1. you may call minlbfgssetxrep(...,true) to turn on reports about algorithm progress, and then you will be able to compare current point with last one reported by algorithm (last point for which algorithm accepted step).

2. it is impossible to return Hessian matrix, algorithm does not evaluate it explicitly.

3. in order to get steepest descent, you may use MinCG subpackage, and manually rewrite it a bit. Find lines below "// Choose BetaK" comment, and rewrite them by "betak = 0.0". It will force CG algorithm to behave like steepest descent.

Or, alternatively, you may add your own value for State.CGType field (you will understand what I mean if you look at this piece of code) - say, CGtype=-1, which will denote "steepest descent" - and add "if( state.cgtype==-1 ) betak = 0;". So you will be able to have both steepest descent and unmodified CG algorithm.
Author:  vigupta [ Tue Nov 05, 2013 4:01 pm ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
Hello,
I understand that the library probably is not computing the Hessian explicitly. However, it will be nice to know what kind of estimation it is using, for example if it is using the Sherman-Morrison-Woodbery to estimate the Hessian.
Quote:
3. in order to get steepest descent, you may use MinCG subpackage, and manually rewrite it a bit. Find lines below "// Choose BetaK" comment, and rewrite them by "betak = 0.0". It will force CG algorithm to behave like steepest descent.

In this case I opened the optimization.cpp I found these lines
Code:
        if( state->cgtype==0 )
        {
            betak = state->betady;
        }
        if( state->cgtype==1 )
        {
            betak = ae_maxreal(0, ae_minreal(state->betady, state->betahs, _state), _state);
        }

I added
Code:
if (state->cgtype == -1)
betak =0;

below this.

However when I run the code it gives me malloc errors
*** glibc detected *** "path": malloc(): memory corruption: 0x0000000001b681a0 ***


Thanks a lot
Vikash
Author:  Sergey.Bochkanov [ Tue Nov 05, 2013 8:07 pm ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
Hmmm.... strange. I have to check it myself. i will get back to you tomorrow.
Author:  vigupta [ Tue Nov 05, 2013 8:17 pm ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
Hi Sergey,
No its fine... I debugged, it was because I was writing
mincgcreate(1, x, state) Now it doesnt give that error anymore..
I am just concerned if setting betak =0 really makes it the classical steepest descent ?

Thanks
Vikash
Author:  Sergey.Bochkanov [ Wed Nov 06, 2013 7:18 am ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
Hello! Thanks for info on the reasons behind failure.

Regarding steepest descent... Yes, it will be classical steepest descent (line search across gradient vector), assuming that: a) betak set to zero, b) preconditioning is turned off (you called minlbfgssetprecdiag() with unit vector as preconditioner).
Author:  vigupta [ Sun Nov 10, 2013 6:14 pm ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
Hi,
I am using minlbfgssetxrep(state ,1) . Still I dont get the step size. Can you tell me where it is stored as in state.xxx which gives the step size ?

Thanks
Vikash
Author:  zohaa3492 [ Mon Nov 10, 2014 5:55 am ]
Post subject:  Re: How do I know what step size the LBFGS algorithm is taki
What compiler you used?
Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/