forum.alglib.net

ALGLIB forum
It is currently Mon Aug 08, 2022 3:56 am

All times are UTC


Forum rules


1. This forum can be used for discussion of both ALGLIB-related and general numerical analysis questions
2. This forum is English-only - postings in other languages will be removed.



Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: Trying to get optimization to work with custom cost function
PostPosted: Tue Sep 25, 2012 8:24 pm 
Offline

Joined: Tue Sep 25, 2012 8:07 pm
Posts: 2
I wish to replace my custom gradient descent algorithm with an optimized algorithm. The solution contains a cost function which can calculate the cost and the gradient.

Code:
public void function1_grad(double[] x, ref double func, double[] grad, object obj)
{
   NNCostFunction costFunction = (NNCostFunction)obj;
   ILArray<double> parameters = ILMath.array(x);
   ResultCostFunction result = costFunction.costFunction(parameters);
   func = result.Cost; //double
   grad = result.Gradient.ToArray(); //double[]
}


One concept I don't understand is why the gradient is not set as output. When trying to optimize the function it runs the functions one time only (while not changing the initial parameters), while the stopping criteria is set to 50 iterations.

Code:
double epsg = 0;
double epsf = 0;
double epsx = 0;
int maxits = 50;
alglib.minlbfgsstate state;
alglib.minlbfgsreport rep;

alglib.minlbfgscreate(x.Length, 3, x,  out state);
alglib.minlbfgssetcond(state, epsg, epsf, epsx, maxits);
alglib.minlbfgsoptimize(state, function1_grad, null, costFunction); //cost function only is called once
alglib.minlbfgsresults(state, out x, out rep);
System.Console.WriteLine(parameters[r(0,20),0]);
parameters.a = ILMath.array(x);
System.Console.WriteLine(parameters[r(0, 20),0]); //results are similar while they should differ


My guess is that the gradient from the cost function is not used. The question is how can I modify my code so it does use the gradient?

Thanks


Top
 Profile  
 
 Post subject: Re: Trying to get optimization to work with custom cost func
PostPosted: Thu Sep 27, 2012 6:43 am 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 878
GrazyMos wrote:
Code:
public void function1_grad(double[] x, ref double func, double[] grad, object obj)
{
   NNCostFunction costFunction = (NNCostFunction)obj;
   ILArray<double> parameters = ILMath.array(x);
   ResultCostFunction result = costFunction.costFunction(parameters);
   func = result.Cost; //double
   grad = result.Gradient.ToArray(); //double[]
}


One concept I don't understand is why the gradient is not set as output. When trying to optimize the function it runs the functions one time only (while not changing the initial parameters), while the stopping criteria is set to 50 iterations.
Thanks

Gradient is not set to out in order to avoid excessive reallocation of dynamic arrays. You have preallocated array grad, and you should write into it - without reallocating its memory. BTW, that's the reason why algorithm stops immediately - you function allocates new array and stores gradient into it, but optimizer is not aware of this change and still looks to the old place (filled with default zeros).

I know that C# programmers tend to think that you can easily allocate dynamic structures due to efficient garbage collector :) But optimizers should be as efficient as possible, and memory allocation is a bottleneck even in C#.


Top
 Profile  
 
 Post subject: Re: Trying to get optimization to work with custom cost func
PostPosted: Thu Sep 27, 2012 9:38 am 
Offline

Joined: Tue Sep 25, 2012 8:07 pm
Posts: 2
Thanks, spot on!

Adding a simple array copy solved the problem
Code:
Array.Copy(result.Gradient.ToArray(),  grad, grad.Length);


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group