forum.alglib.net http://forum.alglib.net/ 

spline1dfit  diverges at dataset edges http://forum.alglib.net/viewtopic.php?f=2&t=4314 
Page 1 of 1 
Author:  pcdangio [ Wed Feb 05, 2020 4:08 am ]  
Post subject:  spline1dfit  diverges at dataset edges  
I'm using alglib's spline1dfit function to smooth some noisy analog electrical signals for characterization purposes. The fitting works very well along the entire window of input data except for at the edges. Check out the attached picture, which shows the spline diverging from the last few data points in both signals. Here's my C++ code for performing the fit: Code: void denoiser::process(const QVector<double> &input_signal, QVector<double> &output_signal) { // Set up X and Y arrays alglib::real_1d_array x, y; // Populate x array. x.setlength(input_signal.size()); for(int32_t i = 0; i < input_signal.size(); i++) { x[i] = i; } // Attach y array to input signal data (shares pointer so no copying) y.attach_to_ptr(input_signal.size(), const_cast<double*>(input_signal.data())); // Set up spline interpolant and fit report structures to capture output. alglib::spline1dinterpolant interpolant; alglib::spline1dfitreport fit_report; // Perform fitting. alglib::spline1dfit(x, y, x.length(), 10*x.length(), denoiser::lambda, interpolant, fit_report); // Populate output with fitted spline. output_signal.clear(); output_signal.reserve(input_signal.size()); for(int32_t i = 0; i < input_signal.size(); i++) { output_signal.append(alglib::spline1dcalc(interpolant, i)); } } The typical data length (N) is between 100 and 500 data points. I've tried using all kinds of values of M, including M = 0.1*N through M = 10*N. It has no effect on the divergence of the spline at the beginning/end of the window. I'm using a lambda of 0.0001. It seems like spline1dfit is simply ignoring the first/last few points when performing the fit, even though I've specified N. Anybody have any ideas on what might be going on here? Thanks!

Author:  pcdangio [ Mon Mar 02, 2020 7:48 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
bump... anybody having similar experiences or know what might be causing the issue? 
Author:  pcdangio [ Mon Apr 06, 2020 3:33 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
update: I've found that the cause is M (number of nodes). If M is too high, it will create the divergence effects seen at the dataset edges. Not sure what makes M "too high", as it seems to depend on the length of the data (N) and the complexity of the data's shape. I'm going to continue testing and will report back anything I find. 
Author:  alex189 [ Thu Nov 12, 2020 2:18 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
Hi, I have the same problem. The generated spline fits well in the "middle", but diverges at the edges. Ideally, I'd like the spline to explicitly go through the start and end point. Is this possible? Maybe by assigning higher weights to start and end point? Thanks in advance! Best regards, Alex 
Author:  newworldguy [ Sat Apr 08, 2023 2:55 pm ]  
Post subject:  Re: spline1dfit  diverges at dataset edges  
I use this function to generate the data: https://www.alglib.net/interpolation/leastsquares.php#splinefit and I get different results for 3.12 and 3.20. Please find attached.

Author:  Sergey.Bochkanov [ Sun Apr 09, 2023 5:53 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
Hi! I confirm that it is a bug  not a mistake in the program, but a problem with the algorithm itself. Actively working on it, it will be included into upcoming release. Sergey. 
Author:  Sergey.Bochkanov [ Thu Apr 20, 2023 7:21 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
The bug is fixed, will be included into upcoming release! 
Author:  ahgu [ Sun Mar 31, 2024 4:35 am ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
Is M same as knots? Where do I specify knots and can I specify the location of knots? Or is this algorithm is independent of knots ? 
Author:  Sergey.Bochkanov [ Sun Mar 31, 2024 7:58 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
M is same as knots, and knots are located on an equidistant grid within an interval spanned by dataset points. In the present version you can not choose knots locations. However, you can choose very fine grid without suffering prohibitive performance penalties (still linear in points/knots count). 
Author:  ahgu [ Sun Mar 31, 2024 8:04 pm ] 
Post subject:  Re: spline1dfit  diverges at dataset edges 
After I get a regression line from running the 1dfit, I need to use the curve. How can I get the least number of parameters and construct the curve? Right now, I can only save the points and reconstruct the curve from points. I wonder if I can save the parameters (The fewest possible) and construct the curve from the parameters analytically. I need to use the curve in a microcontroller, therefore, need fewest number of paramaters as possible. I do see the following function. But don't know the best way to tackle this. l = 0; r = c>n2+1; while(l!=r1) { m = (l+r)/2; if( c>x.ptr.p_double[m]>=x ) { r = m; } else { l = m; } } /* * Interpolation */ x = xc>x.ptr.p_double[l]; m = 4*l; result = c>c.ptr.p_double[m]+x*(c>c.ptr.p_double[m+1]+x*(c>c.ptr.p_double[m+2]+x*c>c.ptr.p_double[m+3])); TRACE(_T("m=%d\n"), m); return result; thank you Andrew 
Page 1 of 1  All times are UTC 
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ 