forum.alglib.net
http://forum.alglib.net/

Doubt in using "lsfitcreatefg"instead of "lsfitcreatef"
http://forum.alglib.net/viewtopic.php?f=2&t=3851
Page 1 of 1

Author:  Dmitrii [ Tue May 15, 2018 2:03 pm ]
Post subject:  Doubt in using "lsfitcreatefg"instead of "lsfitcreatef"

Good day all!

I use C# version of ALGLIB 3.13.0 (source code generated 2017-12-29).

My experimental data are approximated by the sum of the Gaussians (1 - 50) and the background line.

For approximation I use lsfitcreatef method, which works nice for me at least for ~ 30 Gaussians in data. Example of aproximation for 1 Gaussian + line is in the attachement.

To increase the speed and reliability of convergence, I tried to use lsfitcreatefg method but failed.

For clarity, here is my code:

Code:
static void Main(string[] args)
        {
            string filepath = @"D:\data.txt";
            StreamReader reader = new StreamReader(filepath);

            double[] spectr = new double[45];

            for (int counter = 0; counter < spectr.Length; counter++)
            {
                spectr[counter] = Double.Parse(reader.ReadLine());              // y data
            }

            reader.Close();
            reader.Dispose();

            int start_channel = 2107;

            double[,] x = new double[spectr.Length, 1];

              for (int i = 0; i < x.Length; i++)
            {
                x[i, 0] = (double)(i + start_channel);                           // x data
            }

            double[] s = new double[] { 10000, 1000, 1000, 1 };                                                     // scale
            double[] c = new double[] { 18000, 2127, 3747, -3.71 };                                                 // starting values
            double[] bndl = new double[] { 0, 0, Double.NegativeInfinity, Double.NegativeInfinity };               
            double[] bndu = new double[] { +20000, +2150, Double.PositiveInfinity, Double.PositiveInfinity };     

            double epsx = 0.0000001;
            int maxits = 0;
            int info;
            alglib.lsfitstate state;
            alglib.lsfitreport rep;
            double diffstep = 0.000001;

            alglib.lsfitcreatef(x, spectr, c, diffstep, out state);
            alglib.lsfitsetbc(state, bndl, bndu);
            alglib.lsfitsetcond(state, epsx, maxits);
            alglib.lsfitsetscale(state, s);
            alglib.lsfitfit(state, function_func, null, null);
            alglib.lsfitresults(state, out info, out c, out rep);

            System.Console.WriteLine("{0}", info);
            System.Console.WriteLine("{0}", alglib.ap.format(c, 6));

/*          alglib.lsfitcreatefg(x, spectr, c, true, out state);
            alglib.lsfitsetbc(state, bndl, bndu);
            alglib.lsfitsetcond(state, epsx, maxits);
            alglib.lsfitsetscale(state, s);
            alglib.lsfitfit(state, function_func, function_grad, null, null);
            alglib.lsfitresults(state, out info, out c, out rep);
            System.Console.WriteLine("{0}", info);
            System.Console.WriteLine("{0}", alglib.ap.format(c, 6));       */
        }

public static void function_func(double[] c, double[] x, ref double func, object obj)
        {
            func = c[0] * 1 / (Math.Pow(2 * Math.PI, 0.5) * Math.Pow(1.72, 0.5)) * Math.Exp(-(c[1] - x[0]) * (c[1] - x[0]) / (2 * 1.72 * 1.72)) + c[2] + c[3] * x[0] ;
        }

public static void function_grad(double[] c, double[] x, ref double func, double[] grad, object obj)
        {
            grad[0] = 1 / (Math.Pow(2 * Math.PI, 0.5) * Math.Pow(1.72, 0.5)) * Math.Exp(-(c[1] - x[0]) * (c[1] - x[0]) / (2 * 1.72 * 1.72)) + c[2] + c[3] * x[0];
            grad[1] = (c[0] * 1 / (Math.Pow(2 * Math.PI, 0.5) * Math.Pow(1.72, 0.5)) * Math.Exp(-(c[1] - x[0]) * (c[1] - x[0]) / (2 * 1.72 * 1.72)) + c[2] + c[3] * x[0]) *
            ((x[0] - c[1]) / (1.72 * 1.72));
            grad[2] = 1;
            grad[3] = x[0];
        }


For lsfitcreatef method all is fine. Parameters values change from c=[18000, 2127, 3747, -3.71] to c = [10564, 2127, 624, -0.248]. Very good result. Picture with this result is in the attachement.

If I remove the commenting for lsfitcreatefg method in the my code and comment lsfitcreatef method instead the result is c=[17999.99, 2126.99, 3747, -3.71].

The resulting values of the parameters remained practically unchanged.

I checked the accuracy of the computation of gradient functions - all right.

Could you explain what do I do wrong?

Regards

Attachments:
Exp_vs_Approx.PNG
Exp_vs_Approx.PNG [ 38.07 KiB | Viewed 1030 times ]

Author:  Sergey.Bochkanov [ Wed May 16, 2018 10:46 am ]
Post subject:  Re: Doubt in using "lsfitcreatefg"instead of "lsfitcreatef"

Hi!

I noticed two errors:

a) in function_grad() method you should set BOTH grad and func by-ref parameters. Solver expects that you return both function and its gradient upon return from this method.

b) your gradient is calculated incorrectly. It should be dF/dC, and even grad[0] is clearly wrong - it has trailing " + c[2] + c[3] * x[0]" term which should be truncated when you calculate derivative with respect to c[0].

Does it work if you fit ix?

Author:  Dmitrii [ Thu May 24, 2018 11:44 am ]
Post subject:  Re: Doubt in using "lsfitcreatefg"instead of "lsfitcreatef"

Sergey, you were absolutely right. I made stupid mistakes in the calculation of the derivatives, to my shame and sorrow.

With the help of correctly calculated derivatives, everything is worked correctly and 10 times (!) faster (~10 seconds with lsfitcreatef vs ~ 1 second with lsfitcreatefg ).

Thank you very much for the help! I'm sorry that I disturbed you with such an unprepared question.

Sergey, as far as I know, iterative algorithms do not give in to parallelization, or do it very badly. My test sample(1496 data points with 30 Gaussians) is processed in 1 second on Core i7-7700. In this case, only 1 of 8 processor threads is used. Will ALGLIB Commercial Edition process data faster?

May you tell me how to calculate accuracy of estimated model parameters or show the direction of search such information?

In any case, thanks for a good mathematical and software tool!

Author:  Sergey.Bochkanov [ Mon May 28, 2018 9:31 am ]
Post subject:  Re: Doubt in using "lsfitcreatefg"instead of "lsfitcreatef"

Hi, Dmitrii!

I do not think that parallelism will be of any significant help for you. 30 gaussians = somewhat like 150 parameters (in your case). And 150 parameters is not enough to get good speed-up from parallelism.

You may get some speed-up from switching to the native computational core shipped with Commercial Edition (exactly same C# interface). But it is hard to tell how large it will be... I think that 2x at best, but it may be closer to 1.0 - you will spend some additional time communicating between managed and native memory spaces, and this performance penalty can be significant for lightweight problems like yours.

Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/