forum.alglib.net
http://forum.alglib.net/

spline1dfit - diverges at dataset edges
http://forum.alglib.net/viewtopic.php?f=2&t=4314
Page 1 of 1

Author:  pcdangio [ Wed Feb 05, 2020 4:08 am ]
Post subject:  spline1dfit - diverges at dataset edges

I'm using alglib's spline1dfit function to smooth some noisy analog electrical signals for characterization purposes. The fitting works very well along the entire window of input data except for at the edges. Check out the attached picture, which shows the spline diverging from the last few data points in both signals.

Here's my C++ code for performing the fit:

Code:
void denoiser::process(const QVector<double> &input_signal, QVector<double> &output_signal)

    // Set up X and Y arrays
    alglib::real_1d_array x, y;

    // Populate x array.
    x.setlength(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        x[i] = i;
    }

    // Attach y array to input signal data (shares pointer so no copying)
    y.attach_to_ptr(input_signal.size(), const_cast<double*>(input_signal.data()));

    // Set up spline interpolant and fit report structures to capture output.
    alglib::spline1dinterpolant interpolant;
    alglib::spline1dfitreport fit_report;

    // Perform fitting.
    alglib::spline1dfit(x, y, x.length(), 10*x.length(), denoiser::lambda, interpolant, fit_report);

    // Populate output with fitted spline.
    output_signal.clear();
    output_signal.reserve(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        output_signal.append(alglib::spline1dcalc(interpolant, i));
    }
}


The typical data length (N) is between 100 and 500 data points. I've tried using all kinds of values of M, including M = 0.1*N through M = 10*N. It has no effect on the divergence of the spline at the beginning/end of the window. I'm using a lambda of 0.0001.

It seems like spline1dfit is simply ignoring the first/last few points when performing the fit, even though I've specified N.

Anybody have any ideas on what might be going on here? Thanks!

Attachments:
spline_fit.png
spline_fit.png [ 9.71 KiB | Viewed 5877 times ]

Author:  pcdangio [ Mon Mar 02, 2020 7:48 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

bump... anybody having similar experiences or know what might be causing the issue?

Author:  pcdangio [ Mon Apr 06, 2020 3:33 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

update: I've found that the cause is M (number of nodes). If M is too high, it will create the divergence effects seen at the dataset edges. Not sure what makes M "too high", as it seems to depend on the length of the data (N) and the complexity of the data's shape.

I'm going to continue testing and will report back anything I find.

Author:  alex189 [ Thu Nov 12, 2020 2:18 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

Hi,

I have the same problem. The generated spline fits well in the "middle", but diverges at the edges.

Ideally, I'd like the spline to explicitly go through the start and end point.
Is this possible? Maybe by assigning higher weights to start and end point?

Thanks in advance!

Best regards,
Alex

Author:  newworldguy [ Sat Apr 08, 2023 2:55 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

I use this function to generate the data:
https://www.alglib.net/interpolation/leastsquares.php#splinefit


and I get different results for 3.12 and 3.20. Please find attached.

Attachments:
File comment: using 3.20
spline1dfit(x, y, 200, lambdans, s, rep);
lambdans 1.0 / 1000000.0, 1.0 / 10000.0, 1.0 / 1000.0

Figure_1_3.20.png
Figure_1_3.20.png [ 57.3 KiB | Viewed 3488 times ]
File comment: using 3.12
spline1dfitpenalized(x, y, m, rho, info, s, rep); rho values 0,2,6

Figure_1_3.12.png
Figure_1_3.12.png [ 54.59 KiB | Viewed 3488 times ]

Author:  Sergey.Bochkanov [ Sun Apr 09, 2023 5:53 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

Hi!

I confirm that it is a bug - not a mistake in the program, but a problem with the algorithm itself. Actively working on it, it will be included into upcoming release.

Sergey.

Author:  Sergey.Bochkanov [ Thu Apr 20, 2023 7:21 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

The bug is fixed, will be included into upcoming release!

Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/