forum.alglib.net
http://forum.alglib.net/

how to constrain rbf hierarchical output => higherQual.OUTP?
http://forum.alglib.net/viewtopic.php?f=2&t=4570		 Page 1 of 1
Author:  petersesztak [ Tue Jun 06, 2023 4:34 pm ]
Post subject:  how to constrain rbf hierarchical output => higherQual.OUTP?
Hi,

First of all: congratulations for ALGLIB.

I try to use it to solve a Multivariate interpolation.
details: to rbfcreate, rbfsetpoints, rbfsetalgohierarchical, rbfbuildmodel (report = 1), rbfcalc: everything is working perfectly -the documentation and examples well written.
I adjusted the RBase, NumberOfLayers, Lambda according ALGLIB recommendation: ok.

The only problem: I got some very impossible result at at the very end: the evaluation of rbfmodel: rbfcalc when I start moving away from the fixed known points.
(for fixed known points: like a charm !)

More details with two example:
A)
inputs to build RBF model: rbfsetpoints: 300 pairs of measurement 3D spatial data (IN: 3 dimension, eg. RGB color of liquid) and measurement of liquid concentrations of ingredients (allways 8 ingredients, the sum of concentration must be 1!) (OUT dimension 8) .
impossible results at rbfcalc : some of ingredients vector bellow zero (e.g. -0.1234), which is not possible obviously.

B) Same input as example A, but output is a Spectral Reflection measurement with valid measurement range 0 ... 1.

So, my question regarding: how to help the rbf model build to constrain the output within a valid range knew in advance ?
E.g. sum of concentration must be 1 or a certain measurement range must be between 0 .. 1.
(note: the rbfsetpoints do not contain such anomalies, only the result of rbfcalc multi dimension interpolation )

Is there any way to constrain the rbf model build process to limit the output values ?

Thanks for your help in advance,
Best Regards,
Péter
Author:  Sergey.Bochkanov [ Wed Jun 07, 2023 7:34 am ]
Post subject:  Re: how to constrain rbf hierarchical output => higherQual.O
Hi!

That kind of constraints is impossible to achieve with RBFs. Different output components are handled independently, and there is no way to constraint them to sum to 1. It is also impossible to constraint RBF values between [0,1].

You can try ALGLIB neural networks to do the trick. Here "neural network" means "old-fashioned low dimensional approximation algorithm", not "powerful LLM that is smart enough to order a pizza for you" :)
Author:  petersesztak [ Wed Jun 07, 2023 2:11 pm ]
Post subject:  Re: how to constrain rbf hierarchical output => higherQual.O
Dear Sergey !

Thanks for your really valuable info.

I will make a try with ALGLIB neural networks with the following conditions:

alglib.mlptrainer trn;
alglib.multilayerperceptron network;
alglib.mlpreport rep;
Training set: double[,] xy = new double[300, 36 + 8]; // Note: 36 spectral INPUT + 8 Concentrations as OUTPUT = 44
alglib.mlpcreatetrainer(36, 8, out trn); // NIn: 36, NOut: 8
alglib.mlpcreater2(36, 45, 45, 8, 0, 1, out network); // two hidden layer, NHid neurons 8 , 8, constained output [0, 1]
alglib.mlpsetdataset(trn, xy, 300);
alglib.mlptrainnetwork(trn, network, 50, out rep); // // Train Network with 50 nrestarts from random positions
+ EVALUATE with: alglib.mlpprocess(network, x_SPECTRAL, ref y_Concentrations);

And I will report my experience.

If you have any advice, I'd appreciate it (e.g. I chose two hidden layers, 45 - 45 Neurons, 50 random restarts during train).

Thanks in advance,
Péter
Author:  petersesztak [ Wed Jun 07, 2023 3:58 pm ]
Post subject:  Re: how to constrain rbf hierarchical output => higherQual.O
Dear Sergey !

As above mentioned I implemented test code and evaluated random 9 of 300 know pair of validated measurements:

The result improved greatly (individual concentrations never goes below zero) - but not perfect yet (sum of concentration near to 1):

#Eval ∑Concentrations (must be 1)
1 0,954
7 1,017
33 1,033
51 1,021
100 1,003
153 1,044
200 1,041
250 0,924
298 0,995

If I check the details in individual concentration components level: put exactly the same input as model built (see above mentioned Eval #1, 7, .. #298) to mlpprocess and compare the two outputs: original at train set and output of mlpprocess I have some differences, just one examples -see attached picture file.


Attachment:
ALGLIB Sergey mlpprocess one example Concentrations..JPG
ALGLIB Sergey mlpprocess one example Concentrations..JPG [ 46.71 KiB | Viewed 10535 times ]


What do you think : is it the expected level of quality what I can get or is there any tips and tricks / room to improve the quality of the result?

Thanks in advance,
Best Regards,
Péter
Author:  Sergey.Bochkanov [ Thu Jun 08, 2023 4:18 pm ]
Post subject:  Re: how to constrain rbf hierarchical output => higherQual.O
Probably, there is no way to get it with ALGLIB out of the box.

You can fit a model of your own choice, with manually designed structure, using ALGLIB - it has quite good nonlinear optimizers. So, you can incorporate sum-to-one constraints to the model, e.g. via SOFTMAX normalization.
Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/