forum.alglib.net
http://forum.alglib.net/

how to get Levenberg-Marquardt to work
http://forum.alglib.net/viewtopic.php?f=2&t=745		 Page 1 of 1
Author:  BramJ [ Sun Dec 16, 2012 4:13 am ]
Post subject:  how to get Levenberg-Marquardt to work
Hi,

I've created a small wrapper in F# for the Levenberg-Marquardt alglib interface. Upon testing with simple functions, it seems to work. However, upon a real-life data set I encounter problems, which lead me to believe that this is due to me not understanding what is going on with the optimizer rather than my wrapper.

Basically my wrapper is a small class that where there are a few tolerances to be set and then a bunch of overloaded "Minimize" methods. The one I am using is:

member this.Minimize (n, (f : float [] -> float), lb, ub, ?startValue) =
let startValue = defaultArg startValue [| for i in 1 .. n do yield 0.|]
let x = ref startValue
let lb = ref lb
let ub = ref ub

alglib.minlmcreatev(1, !x, dstep, state)
alglib.minlmsetcond(!state, epsg, epsf, epsx, maxits)
alglib.minlmsetbc(!state, !lb, !ub)
alglib.minlmoptimize(!state, (fun xa y o -> y.[0] <- (f xa) ), null, null)
alglib.minlmresults(!state, x, rep)

!rep, !x

In particular, I've got a function which takes 40 input parameters and produces a log likelihood function given those values. When I call this:

let r, sol = LM.Minimize(paramCount, targetAlglibFunc, lowerBound, upperBound, allParams)

with
let targetAlglibFunc (x : float []) =
let a= x.[0..tCount-1]
let d = x.[tCount..2*tCount-1]
let rho = x.[paramCount-2]
let hf = x.[paramCount-1]
-(pseudoLogLikelihoodData data t dffunc a d rho hf)

then I get as a result:
alglib+minlmreport {funcidx = -1;
innerobj = alglib+minlm+minlmreport;
iterationscount = 0;
ncholesky = 0;
nfunc = 81;
ngrad = 0;
nhess = 0;
njac = 1;
terminationtype = 4;
varidx = -1;

which seems to indicate that the numerical gradient is 0. However, when evaluating the targetAlglibFunc at typical values and then shifting these by 1% or so, I do get changes in the pseudologlikihood function (a typical order of magnitude here is ~2000 and shifting all input parameters by 1% might shift that value to 5 points or so).

Is there anything obviously/glaringly wrong here?
Author:  Sergey.Bochkanov [ Sun Dec 16, 2012 3:06 pm ]
Post subject:  Re: how to get Levenberg-Marquardt to work
Can you post actual code here? It is hard to debug algorithm without looking at code.
Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/