forum.alglib.net

Hello!

I read that ALGLIB does not support multithreading.

What does this mean?
Does it mean that ALGLIB does not make use of multithreading to speed up calculations? Or does it mean that it is impossible to execute several ALGLIB calculations in parallel threads?

I am not a computer scientist and would appreciate if someone could give a short explanation on this topic.

Kind regards,
Christian

First one. ALGLIB can't use multiple cores, but nothing prevents you from running several threads (manually) and calling ALGLIB functions from them. Each function will work separately from others, called in parallel, as long as you share no data between them. See example below.

Thank you. That is what I want to do.

Hello, friends!

Probably I repeat topic question but I don’t know was there NNE (Neural networks ensemble) in Alglib when this theme was created.

I use functions: alglib::mlpecreatec1(), alglib::mlpetraines(), alglib::mlpeserialize(), alglib::mlpeunserialize(), alglib::mlpeprocess(), alglib_impl::_mlpensemble_clear() in my code and I am going to trains several different separate neural network ensembles simultaneously in several threads running manually.

I hope NNE functions are able to work separately from others, called in parallel, so as all other Alglib functions ?

Waiting for a reply. Thank you in advance.
Kind regards,
Kirill

P.S. I use Alglib Free Edition v.3.8.2 (C++) now and intend to migrate to 3.10.0.

As long as different threads do not share ALGLIB objects, it is completely safe - ALGLIB has no global thread-unsafe variables.

Thank you a lot! You have dispelled my doubts.

Dear Sergey!

My task is regression analisys. I have some hundreds of data sets and I train the same quantity of neural networks essembles on them (function mlpetraines() ).

There are the same quantity of input vectors for each data set of course. With function mlpeprocess() I define the result for each vector. Finally I define the maximum of all the results.

I use multi-threaded calculation using the CreateThread() function in C++. And depend on the numbers of using threads the results were different! Although the data sets and the vectors were equals! The different was the order of calculations only.

I assume that the reason is the different initial network weights that the function mlpecreate1() sets to random values. I made an experiment. In single thread I twice call mlpetraines() with the same data set (training set) and mlpeprocess() with the same vector and have got the different results:


Result_2 is not equal to Result_1. Ok, this only confirms my assumption.

To order to make the initial random values of the neural net weights the same I remember ensemble_A immediately after function mlpecreate1() the first call and then restore them before the second call of mlpetraines():

Result_2 is not equal to Result_1 again!
I do not understand anything. It looks like if one wants to calculate sinus of the some value twice and have got different results. Is not it ?

Is it possible to train the ensemble with the same data twice in the single code space and get the same trained ensemble in result? How do you achieve that effect and what is it – a bug or a feature ?

Hello!

Training functions re-initialize network by random values every time you call them. So, training network twice on same results will return two different results - it is feature, not bug. If you want, you may comment call to mlprandomize() in mlptraines() (not mlpetraines!), it will make training deterministic.

Hello, Sergey!
Everything is clear now. However it seems to me that to comment the call of mlprandomize() in mlptraines() is hardly good idea because every restart of the training will be without any initial sets of network in this case. Am I not right ?

hmmm... yes, you are right! From the other side...

if you really need to get deterministic behavior, you may seed system RNG with srand(12345) prior to calling training function. However, effect of this call will be long-lasting and may influence other parts of your program.