forum.alglib.net

ALGLIB forum
 It is currently Tue Nov 29, 2022 3:25 pm

 All times are UTC

Forum rules

1. This forum can be used for discussion of both ALGLIB-related and general numerical analysis questions
2. This forum is English-only - postings in other languages will be removed.

 Page 1 of 1 [ 3 posts ]
 Print view Previous topic | Next topic
Author Message
 Post subject: Trying to get optimization to work with custom cost functionPosted: Tue Sep 25, 2012 8:24 pm

Joined: Tue Sep 25, 2012 8:07 pm
Posts: 2
I wish to replace my custom gradient descent algorithm with an optimized algorithm. The solution contains a cost function which can calculate the cost and the gradient.

Code:
{
NNCostFunction costFunction = (NNCostFunction)obj;
ILArray<double> parameters = ILMath.array(x);
ResultCostFunction result = costFunction.costFunction(parameters);
func = result.Cost; //double
}

One concept I don't understand is why the gradient is not set as output. When trying to optimize the function it runs the functions one time only (while not changing the initial parameters), while the stopping criteria is set to 50 iterations.

Code:
double epsg = 0;
double epsf = 0;
double epsx = 0;
int maxits = 50;
alglib.minlbfgsstate state;
alglib.minlbfgsreport rep;

alglib.minlbfgscreate(x.Length, 3, x,  out state);
alglib.minlbfgssetcond(state, epsg, epsf, epsx, maxits);
alglib.minlbfgsoptimize(state, function1_grad, null, costFunction); //cost function only is called once
alglib.minlbfgsresults(state, out x, out rep);
System.Console.WriteLine(parameters[r(0,20),0]);
parameters.a = ILMath.array(x);
System.Console.WriteLine(parameters[r(0, 20),0]); //results are similar while they should differ

My guess is that the gradient from the cost function is not used. The question is how can I modify my code so it does use the gradient?

Thanks

Top

 Post subject: Re: Trying to get optimization to work with custom cost funcPosted: Thu Sep 27, 2012 6:43 am

Joined: Fri May 07, 2010 7:06 am
Posts: 880
GrazyMos wrote:
Code:
{
NNCostFunction costFunction = (NNCostFunction)obj;
ILArray<double> parameters = ILMath.array(x);
ResultCostFunction result = costFunction.costFunction(parameters);
func = result.Cost; //double
}

One concept I don't understand is why the gradient is not set as output. When trying to optimize the function it runs the functions one time only (while not changing the initial parameters), while the stopping criteria is set to 50 iterations.
Thanks

Gradient is not set to out in order to avoid excessive reallocation of dynamic arrays. You have preallocated array grad, and you should write into it - without reallocating its memory. BTW, that's the reason why algorithm stops immediately - you function allocates new array and stores gradient into it, but optimizer is not aware of this change and still looks to the old place (filled with default zeros).

I know that C# programmers tend to think that you can easily allocate dynamic structures due to efficient garbage collector :) But optimizers should be as efficient as possible, and memory allocation is a bottleneck even in C#.

Top

 Post subject: Re: Trying to get optimization to work with custom cost funcPosted: Thu Sep 27, 2012 9:38 am

Joined: Tue Sep 25, 2012 8:07 pm
Posts: 2
Thanks, spot on!

Adding a simple array copy solved the problem
Code:

Top

 Display posts from previous: All posts1 day7 days2 weeks1 month3 months6 months1 year Sort by AuthorPost timeSubject AscendingDescending
 Page 1 of 1 [ 3 posts ]

 All times are UTC

Who is online

Users browsing this forum: No registered users and 2 guests

 You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum

Search for: