forum.alglib.net
http://forum.alglib.net/

spline1dfit - diverges at dataset edges
http://forum.alglib.net/viewtopic.php?f=2&t=4314
Page 1 of 1

Author:  pcdangio [ Wed Feb 05, 2020 4:08 am ]
Post subject:  spline1dfit - diverges at dataset edges

I'm using alglib's spline1dfit function to smooth some noisy analog electrical signals for characterization purposes. The fitting works very well along the entire window of input data except for at the edges. Check out the attached picture, which shows the spline diverging from the last few data points in both signals.

Here's my C++ code for performing the fit:

Code:
void denoiser::process(const QVector<double> &input_signal, QVector<double> &output_signal)

    // Set up X and Y arrays
    alglib::real_1d_array x, y;

    // Populate x array.
    x.setlength(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        x[i] = i;
    }

    // Attach y array to input signal data (shares pointer so no copying)
    y.attach_to_ptr(input_signal.size(), const_cast<double*>(input_signal.data()));

    // Set up spline interpolant and fit report structures to capture output.
    alglib::spline1dinterpolant interpolant;
    alglib::spline1dfitreport fit_report;

    // Perform fitting.
    alglib::spline1dfit(x, y, x.length(), 10*x.length(), denoiser::lambda, interpolant, fit_report);

    // Populate output with fitted spline.
    output_signal.clear();
    output_signal.reserve(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        output_signal.append(alglib::spline1dcalc(interpolant, i));
    }
}


The typical data length (N) is between 100 and 500 data points. I've tried using all kinds of values of M, including M = 0.1*N through M = 10*N. It has no effect on the divergence of the spline at the beginning/end of the window. I'm using a lambda of 0.0001.

It seems like spline1dfit is simply ignoring the first/last few points when performing the fit, even though I've specified N.

Anybody have any ideas on what might be going on here? Thanks!

Attachments:
spline_fit.png
spline_fit.png [ 9.71 KiB | Viewed 6071 times ]

Author:  pcdangio [ Mon Mar 02, 2020 7:48 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

bump... anybody having similar experiences or know what might be causing the issue?

Author:  pcdangio [ Mon Apr 06, 2020 3:33 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

update: I've found that the cause is M (number of nodes). If M is too high, it will create the divergence effects seen at the dataset edges. Not sure what makes M "too high", as it seems to depend on the length of the data (N) and the complexity of the data's shape.

I'm going to continue testing and will report back anything I find.

Author:  alex189 [ Thu Nov 12, 2020 2:18 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

Hi,

I have the same problem. The generated spline fits well in the "middle", but diverges at the edges.

Ideally, I'd like the spline to explicitly go through the start and end point.
Is this possible? Maybe by assigning higher weights to start and end point?

Thanks in advance!

Best regards,
Alex

Author:  newworldguy [ Sat Apr 08, 2023 2:55 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

I use this function to generate the data:
https://www.alglib.net/interpolation/leastsquares.php#splinefit


and I get different results for 3.12 and 3.20. Please find attached.

Attachments:
File comment: using 3.20
spline1dfit(x, y, 200, lambdans, s, rep);
lambdans 1.0 / 1000000.0, 1.0 / 10000.0, 1.0 / 1000.0

Figure_1_3.20.png
Figure_1_3.20.png [ 57.3 KiB | Viewed 3682 times ]
File comment: using 3.12
spline1dfitpenalized(x, y, m, rho, info, s, rep); rho values 0,2,6

Figure_1_3.12.png
Figure_1_3.12.png [ 54.59 KiB | Viewed 3682 times ]

Author:  Sergey.Bochkanov [ Sun Apr 09, 2023 5:53 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

Hi!

I confirm that it is a bug - not a mistake in the program, but a problem with the algorithm itself. Actively working on it, it will be included into upcoming release.

Sergey.

Author:  Sergey.Bochkanov [ Thu Apr 20, 2023 7:21 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

The bug is fixed, will be included into upcoming release!

Author:  ahgu [ Sun Mar 31, 2024 4:35 am ]
Post subject:  Re: spline1dfit - diverges at dataset edges

Is M same as knots?
Where do I specify knots and can I specify the location of knots? Or is this algorithm is independent of knots ?

Author:  Sergey.Bochkanov [ Sun Mar 31, 2024 7:58 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

M is same as knots, and knots are located on an equidistant grid within an interval spanned by dataset points. In the present version you can not choose knots locations. However, you can choose very fine grid without suffering prohibitive performance penalties (still linear in points/knots count).

Author:  ahgu [ Sun Mar 31, 2024 8:04 pm ]
Post subject:  Re: spline1dfit - diverges at dataset edges

After I get a regression line from running the 1dfit, I need to use the curve.
How can I get the least number of parameters and construct the curve? Right now, I can only save the points and re-construct the curve from points. I wonder if I can save the parameters (The fewest possible) and construct the curve from the parameters analytically.

I need to use the curve in a microcontroller, therefore, need fewest number of paramaters as possible.

I do see the following function. But don't know the best way to tackle this.


l = 0;
r = c->n-2+1;
while(l!=r-1)
{
m = (l+r)/2;
if( c->x.ptr.p_double[m]>=x )
{
r = m;
}
else
{
l = m;
}
}

/*
* Interpolation
*/
x = x-c->x.ptr.p_double[l];
m = 4*l;
result = c->c.ptr.p_double[m]+x*(c->c.ptr.p_double[m+1]+x*(c->c.ptr.p_double[m+2]+x*c->c.ptr.p_double[m+3]));
TRACE(_T("m=%d\n"), m);
return result;

thank you
Andrew

Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/