forum.alglib.net

ALGLIB forum
It is currently Sun Nov 03, 2024 12:24 am

All times are UTC


Forum rules


1. This forum can be used for discussion of both ALGLIB-related and general numerical analysis questions
2. This forum is English-only - postings in other languages will be removed.



Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: spline1dfit - diverges at dataset edges
PostPosted: Wed Feb 05, 2020 4:08 am 
Offline

Joined: Wed Feb 05, 2020 3:54 am
Posts: 3
I'm using alglib's spline1dfit function to smooth some noisy analog electrical signals for characterization purposes. The fitting works very well along the entire window of input data except for at the edges. Check out the attached picture, which shows the spline diverging from the last few data points in both signals.

Here's my C++ code for performing the fit:

Code:
void denoiser::process(const QVector<double> &input_signal, QVector<double> &output_signal)

    // Set up X and Y arrays
    alglib::real_1d_array x, y;

    // Populate x array.
    x.setlength(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        x[i] = i;
    }

    // Attach y array to input signal data (shares pointer so no copying)
    y.attach_to_ptr(input_signal.size(), const_cast<double*>(input_signal.data()));

    // Set up spline interpolant and fit report structures to capture output.
    alglib::spline1dinterpolant interpolant;
    alglib::spline1dfitreport fit_report;

    // Perform fitting.
    alglib::spline1dfit(x, y, x.length(), 10*x.length(), denoiser::lambda, interpolant, fit_report);

    // Populate output with fitted spline.
    output_signal.clear();
    output_signal.reserve(input_signal.size());
    for(int32_t i = 0; i < input_signal.size(); i++)
    {
        output_signal.append(alglib::spline1dcalc(interpolant, i));
    }
}


The typical data length (N) is between 100 and 500 data points. I've tried using all kinds of values of M, including M = 0.1*N through M = 10*N. It has no effect on the divergence of the spline at the beginning/end of the window. I'm using a lambda of 0.0001.

It seems like spline1dfit is simply ignoring the first/last few points when performing the fit, even though I've specified N.

Anybody have any ideas on what might be going on here? Thanks!


Attachments:
spline_fit.png
spline_fit.png [ 9.71 KiB | Viewed 20468 times ]
Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Mon Mar 02, 2020 7:48 pm 
Offline

Joined: Wed Feb 05, 2020 3:54 am
Posts: 3
bump... anybody having similar experiences or know what might be causing the issue?


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Mon Apr 06, 2020 3:33 pm 
Offline

Joined: Wed Feb 05, 2020 3:54 am
Posts: 3
update: I've found that the cause is M (number of nodes). If M is too high, it will create the divergence effects seen at the dataset edges. Not sure what makes M "too high", as it seems to depend on the length of the data (N) and the complexity of the data's shape.

I'm going to continue testing and will report back anything I find.


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Thu Nov 12, 2020 2:18 pm 
Offline

Joined: Thu Nov 12, 2020 2:13 pm
Posts: 1
Hi,

I have the same problem. The generated spline fits well in the "middle", but diverges at the edges.

Ideally, I'd like the spline to explicitly go through the start and end point.
Is this possible? Maybe by assigning higher weights to start and end point?

Thanks in advance!

Best regards,
Alex


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Sat Apr 08, 2023 2:55 pm 
Offline

Joined: Sat Apr 08, 2023 2:49 pm
Posts: 1
I use this function to generate the data:
https://www.alglib.net/interpolation/leastsquares.php#splinefit


and I get different results for 3.12 and 3.20. Please find attached.


Attachments:
File comment: using 3.20
spline1dfit(x, y, 200, lambdans, s, rep);
lambdans 1.0 / 1000000.0, 1.0 / 10000.0, 1.0 / 1000.0

Figure_1_3.20.png
Figure_1_3.20.png [ 57.3 KiB | Viewed 18079 times ]
File comment: using 3.12
spline1dfitpenalized(x, y, m, rho, info, s, rep); rho values 0,2,6

Figure_1_3.12.png
Figure_1_3.12.png [ 54.59 KiB | Viewed 18079 times ]
Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Sun Apr 09, 2023 5:53 pm 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 925
Hi!

I confirm that it is a bug - not a mistake in the program, but a problem with the algorithm itself. Actively working on it, it will be included into upcoming release.

Sergey.


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Thu Apr 20, 2023 7:21 pm 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 925
The bug is fixed, will be included into upcoming release!


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Sun Mar 31, 2024 4:35 am 
Offline

Joined: Sun Mar 31, 2024 4:32 am
Posts: 2
Is M same as knots?
Where do I specify knots and can I specify the location of knots? Or is this algorithm is independent of knots ?


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Sun Mar 31, 2024 7:58 pm 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 925
M is same as knots, and knots are located on an equidistant grid within an interval spanned by dataset points. In the present version you can not choose knots locations. However, you can choose very fine grid without suffering prohibitive performance penalties (still linear in points/knots count).


Top
 Profile  
 
 Post subject: Re: spline1dfit - diverges at dataset edges
PostPosted: Sun Mar 31, 2024 8:04 pm 
Offline

Joined: Sun Mar 31, 2024 4:32 am
Posts: 2
After I get a regression line from running the 1dfit, I need to use the curve.
How can I get the least number of parameters and construct the curve? Right now, I can only save the points and re-construct the curve from points. I wonder if I can save the parameters (The fewest possible) and construct the curve from the parameters analytically.

I need to use the curve in a microcontroller, therefore, need fewest number of paramaters as possible.

I do see the following function. But don't know the best way to tackle this.


l = 0;
r = c->n-2+1;
while(l!=r-1)
{
m = (l+r)/2;
if( c->x.ptr.p_double[m]>=x )
{
r = m;
}
else
{
l = m;
}
}

/*
* Interpolation
*/
x = x-c->x.ptr.p_double[l];
m = 4*l;
result = c->c.ptr.p_double[m]+x*(c->c.ptr.p_double[m+1]+x*(c->c.ptr.p_double[m+2]+x*c->c.ptr.p_double[m+3]));
TRACE(_T("m=%d\n"), m);
return result;

thank you
Andrew


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC


Who is online

Users browsing this forum: Bing [Bot] and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group