forum.alglib.net

ALGLIB forum
It is currently Sun Dec 04, 2022 4:18 pm

All times are UTC


Forum rules


1. This forum can be used for discussion of both ALGLIB-related and general numerical analysis questions
2. This forum is English-only - postings in other languages will be removed.



Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: ALGLIB and multithreading
PostPosted: Fri Sep 17, 2010 11:24 am 
Offline

Joined: Fri Sep 17, 2010 11:15 am
Posts: 2
Hello!

I read that ALGLIB does not support multithreading.

What does this mean?
Does it mean that ALGLIB does not make use of multithreading to speed up calculations? Or does it mean that it is impossible to execute several ALGLIB calculations in parallel threads?

I am not a computer scientist and would appreciate if someone could give a short explanation on this topic.

Kind regards,
Christian


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Fri Sep 17, 2010 12:44 pm 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 880
First one. ALGLIB can't use multiple cores, but nothing prevents you from running several threads (manually) and calling ALGLIB functions from them. Each function will work separately from others, called in parallel, as long as you share no data between them. See example below.

Code:
thread1:
    alglib_function1(matrix1)

thread2:
    alglib_function1(matrix2)

everything OK, first thread works with matrix1, second one works with matrix2. No data shared between threads, no problems.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Fri Sep 17, 2010 2:26 pm 
Offline

Joined: Fri Sep 17, 2010 11:15 am
Posts: 2
Thank you. That is what I want to do.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Wed Jun 29, 2016 10:51 am 
Offline

Joined: Mon Dec 09, 2013 12:38 pm
Posts: 7
Hello, friends!

Probably I repeat topic question but I don’t know was there NNE (Neural networks ensemble) in Alglib when this theme was created.

I use functions: alglib::mlpecreatec1(), alglib::mlpetraines(), alglib::mlpeserialize(), alglib::mlpeunserialize(), alglib::mlpeprocess(), alglib_impl::_mlpensemble_clear() in my code and I am going to trains several different separate neural network ensembles simultaneously in several threads running manually.

I hope NNE functions are able to work separately from others, called in parallel, so as all other Alglib functions ?

Waiting for a reply. Thank you in advance.
Kind regards,
Kirill

P.S. I use Alglib Free Edition v.3.8.2 (C++) now and intend to migrate to 3.10.0.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Thu Jun 30, 2016 10:33 am 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 880
As long as different threads do not share ALGLIB objects, it is completely safe - ALGLIB has no global thread-unsafe variables.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Thu Jun 30, 2016 10:47 am 
Offline

Joined: Mon Dec 09, 2013 12:38 pm
Posts: 7
Thank you a lot! You have dispelled my doubts.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Sun Jul 03, 2016 2:23 pm 
Offline

Joined: Mon Dec 09, 2013 12:38 pm
Posts: 7
Dear Sergey!

My task is regression analisys. I have some hundreds of data sets and I train the same quantity of neural networks essembles on them (function mlpetraines() ).

There are the same quantity of input vectors for each data set of course. With function mlpeprocess() I define the result for each vector. Finally I define the maximum of all the results.

I use multi-threaded calculation using the CreateThread() function in C++. And depend on the numbers of using threads the results were different! Although the data sets and the vectors were equals! The different was the order of calculations only.

I assume that the reason is the different initial network weights that the function mlpecreate1() sets to random values. I made an experiment. In single thread I twice call mlpetraines() with the same data set (training set) and mlpeprocess() with the same vector and have got the different results:

Code:
alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);
alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);
alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);
Result_1 = y_Out(0);

// Repeat the same code

alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);
alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);
alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);
Result_2 = y_Out(0);

Result_2 is not equal to Result_1. Ok, this only confirms my assumption.

To order to make the initial random values of the neural net weights the same I remember ensemble_A immediately after function mlpecreate1() the first call and then restore them before the second call of mlpetraines():
Code:
alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);
alglib::mlpeserialize(ensemble_A, string_mem);
alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);
alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);
Result_1 = y_Out(0);

alglib::mlpeunserialize(string_mem, ensemble_A);
alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);
alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);
Result_2 = y_Out(0);

Result_2 is not equal to Result_1 again!
I do not understand anything. It looks like if one wants to calculate sinus of the some value twice and have got different results. Is not it ?

Is it possible to train the ensemble with the same data twice in the single code space and get the same trained ensemble in result? How do you achieve that effect and what is it – a bug or a feature ?


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Mon Jul 04, 2016 9:00 am 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 880
Hello!

Training functions re-initialize network by random values every time you call them. So, training network twice on same results will return two different results - it is feature, not bug. If you want, you may comment call to mlprandomize() in mlptraines() (not mlpetraines!), it will make training deterministic.


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Mon Jul 04, 2016 9:43 am 
Offline

Joined: Mon Dec 09, 2013 12:38 pm
Posts: 7
Hello, Sergey!
Everything is clear now. However it seems to me that to comment the call of mlprandomize() in mlptraines() is hardly good idea because every restart of the training will be without any initial sets of network in this case. Am I not right ?


Top
 Profile  
 
 Post subject: Re: ALGLIB and multithreading
PostPosted: Mon Jul 04, 2016 2:25 pm 
Offline
Site Admin

Joined: Fri May 07, 2010 7:06 am
Posts: 880
hmmm... yes, you are right! From the other side...

if you really need to get deterministic behavior, you may seed system RNG with srand(12345) prior to calling training function. However, effect of this call will be long-lasting and may influence other parts of your program.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group